Mobile terminal

ABSTRACT

Disclosed is a mobile terminal comprising: a communication unit for communicating with a plurality of target devices in a Bluetooth Low Energy (BLE) manner; a camera for capturing an image; and a controller configured for: receiving a plurality of advertisement information respectively including a plurality of shape information of the plurality of target devices from the plurality of target devices; and when first and second target devices among the plurality of target devices have the same shape information and when a distance of the first target device is equal to a distance of the second target device, activating a connection between the mobile terminal and the first target device based on an operation of at least one of the first target device or the second target device.

BACKGROUND

The present disclosure relates to a terminal capable of photographing atarget device to be connected to establish a short-range wirelesscommunication connection, such as BLUETOOTH low energy (BLE) connection,with the target device.

BLUETOOTH is a short-range wireless technology standard for wirelesslyconnecting various devices to exchange data over short distances. Inorder to perform wireless communication between two devices usingBLUETOOTH communication, a user performs a procedure of discoveringBLUETOOTH devices for communication and requesting a connection. In thepresent disclosure, the device may mean equipment or an apparatus.

At this time, the user may discover and then connect the BLUETOOTHdevices according to a BLUETOOTH communication method.

The BLUETOOTH communication method includes a basic rate/enhanced datarate (BR/EDR) method and a low energy (LE) method. BR/EDR may also bereferred to as BLUETOOTH Classic. The BLUETOOTH classic method includesBLUETOOTH technology which has been used since BLUETOOTH version number1.0 using a basic rate and BLUETOOTH technology using an enhanced datarate which has been supported since BLUETOOTH version number 2.0.

BLUETOOTH low energy (hereinafter, referred to as BLUETOOTH LE)technology has been applied since BLUETOOTH version number 4.0 and maystably provide information of several hundred kilobytes (kB) with lowpower consumption. Such BLUETOOTH LE technology uses an attributeprotocol to enable information exchange between devices. Such BLUETOOTHLE technology may reduce header overhead and simplify operation, therebyreducing energy consumption.

Some BLUETOOTH devices may not include a display or a user interface.Complexity of connection/management/control/disconnection betweenvarious types of BLUETOOTH devices and, more particularly, BLUETOOTHdevices using similar technologies is increasing.

In addition, BLUETOOTH has a relatively high data rate with relativelylow power and low cost, but a transmission distance is typically limitedto a maximum of 100 m. Therefore, BLUETOOTH is most suitable for use inconfined spaces and across relatively short-range distances.

Meanwhile, when a terminal and another device are to be connected usinga conventional BLUETOOTH LE method, many processes or operations have tobe performed for pairing and connection.

For example, the user must initiate a scan after entering a BLUETOOTHmenu, confirm and select a device to be connected in a list of numerousscanned and discovered devices, and initiate a connection to establish apairing and connection. Only after the connection is establishedfollowing the numerous connection operations, the user is then able toexecute a dedicated application (“app”) to control the selected device.

SUMMARY

The present disclosure relates to a terminal capable of photographing atarget device to be connected to establish a short-range wirelesscommunication connection, such as BLUETOOTH low energy (BLE) connection,with the target device.

In one aspect, the present disclosure provides a mobile terminalcomprising: a communication unit for communicating with a plurality oftarget devices in a Bluetooth Low Energy (BLE) manner; a camera forcapturing an image; and a controller configured for: receiving aplurality of advertisement information respectively including aplurality of shape information of the plurality of target devices fromthe plurality of target devices; and when first and second targetdevices among the plurality of target devices have the same shapeinformation and when a distance of the first target device is equal to adistance of the second target device, activating a connection betweenthe mobile terminal and the first target device based on an operation ofat least one of the first target device or the second target device.

BRIEF DESCRIPTIONS OF DRAWINGS

FIG. 1 shows an example of a wireless communication system usingBLUETOOTH low energy technology according to an embodiment of thepresent disclosure.

FIGS. 2A, 2B, 2C, and 2D are diagrams illustrating examples of a BLEconnection.

FIG. 3 is a block diagram illustrating a terminal 100 according to anembodiment of the present disclosure.

FIG. 4 is a diagram illustrating a method of operating a wirelesscommunication system according to an embodiment of the presentdisclosure.

FIG. 5 is a diagram illustrating a process of establishing a connectionbetween a terminal and a first target device.

FIG. 6 is a diagram showing an advertisement packet transmitted by afirst target device.

FIG. 7 is a diagram showing a learning process of a machine learningmodel.

FIG. 8 is a diagram showing a state in which a target device isphotographed by a terminal.

FIG. 9 is a diagram illustrating a method of generating a list ofidentification information estimated using a photographed imageaccording to an embodiment of the present disclosure.

FIGS. 10 and 11 are diagrams illustrating a method of selecting a targetdevice for connection and control using estimated identificationinformation and identification information of a plurality of targetdevices acquired by scanning, according to an embodiment of the presentdisclosure.

FIG. 12 is a diagram showing a terminal 100 and a first target deviceaccording to an embodiment of the present disclosure.

FIG. 13 is another diagram showing a terminal 100 according to anembodiment of the present disclosure.

FIG. 14 is a diagram showing state change of a target device accordingto an embodiment of the present disclosure.

FIG. 15 is a diagram showing state change of a terminal according to anembodiment of the present disclosure.

FIGS. 16A, 16B, and 16C are diagrams showing a process of establishingconnection with a first target device according to an embodiment of thepresent disclosure.

FIG. 17 is a diagram showing a control icon corresponding to thefunction of a target device according to an embodiment of the presentdisclosure.

FIGS. 18A, 18B, and 18C are diagrams illustrating a method of inputtinga voice command using a terminal according to an embodiment of thepresent disclosure.

FIG. 19 is a diagram illustrating a wireless communication systemaccording to another embodiment of the present disclosure.

FIG. 20 is a diagram showing a terminal communicating with a pluralityof target devices according to an embodiment of the present disclosure.

FIG. 21 is a diagram illustrating a method of operating a wirelesscommunication system according to an embodiment of the presentdisclosure.

FIG. 22 is a diagram showing an example of the data format of anadvertisement packet according to an embodiment of the presentdisclosure.

FIG. 23 is a diagram showing a method of estimating shape informationusing a photographed image according to an embodiment of the presentdisclosure.

FIG. 24 is a diagram illustrating an example of a method of acquiring adistance from a target device according to an embodiment of the presentdisclosure.

FIG. 25 is a diagram showing a list of detailed information according toan embodiment of the present disclosure.

FIG. 26 is a diagram illustrating advertisement information includingonly shape information.

FIG. 27 is a diagram showing a list of advertisement informationreceived from three target devices according to an embodiment of thepresent disclosure.

FIG. 28 is a diagram illustrating a method of operating a wirelesscommunication system according to an embodiment of the presentdisclosure.

FIG. 29 is a diagram showing a detailed information list according to anembodiment of the present disclosure.

FIGS. 30 and 31 are diagrams showing a list of searched detailedinformation.

FIG. 32 is a diagram showing information included in a detailedinformation list and information acquired by a controller according toan embodiment of the present disclosure.

FIG. 33 is a view illustrating a method of establishing a connectionwith a target device according to the embodiment of the presentdisclosure.

FIG. 34 illustrates an operating method of a wireless communicationsystem according to an embodiment of the present disclosure.

FIG. 35 illustrates a method of activating a connection with a targetdevice when the user is illuminating a target device, according to anembodiment of the present disclosure.

FIG. 36 illustrates a method for activating a connection with a targetdevice when the user is illuminating two target devices, according to anembodiment of the present disclosure.

FIG. 37 illustrates a method of activating a connection with any onetarget device when the user is illuminating two target devices,according to an embodiment of the present disclosure.

FIG. 38 illustrates an operating method of a wireless communicationsystem according to another embodiment of the present disclosure.

FIG. 39 illustrates a method of activating a connection with a targetdevice when the user is illuminating one target device, according to anembodiment of the present disclosure.

FIG. 40 is a diagram illustrating a method for activating a connectionwith a target device when the user is illuminating two target devices,according to an embodiment of the present disclosure.

FIG. 41 is a diagram for explaining a method of identifying a targetdevice using an image pattern, according to an embodiment of the presentdisclosure.

FIG. 42 illustrates an operating method of a wireless communicationsystem according to another embodiment of the present disclosure.

FIG. 43 illustrates a method of activating a connection with a targetdevice when the user is illuminating one target device, according to anembodiment of the present disclosure.

FIG. 44 illustrates a method for activating a connection with one of twotarget devices according to an embodiment of the present disclosure.

DETAILED DESCRIPTIONS

Description will now be given in detail according to exemplaryembodiments disclosed herein, with reference to the accompanyingdrawings. For the sake of brief description with reference to thedrawings, the same or equivalent components may be provided with thesame reference numbers, and description thereof will not be repeated. Ingeneral, a suffix such as “module” and “unit” may be used to refer toelements or components. Use of such a suffix herein is merely intendedto facilitate description of the specification, and the suffix itself isnot intended to give any special meaning or function. In the presentdisclosure, that which is well-known to one of ordinary skill in therelevant art has generally been omitted for the sake of brevity. Theaccompanying drawings are used to help easily understand varioustechnical features and it should be understood that the embodimentspresented herein are not limited by the accompanying drawings. As such,the present disclosure should be construed to extend to any alterations,equivalents and substitutes in addition to those which are particularlyset out in the accompanying drawings.

It will be understood that although the terms first, second, etc. may beused herein to describe various elements, these elements should not belimited by these terms. These terms are generally only used todistinguish one element from another.

It will be understood that when an element is referred to as being“connected with” another element, the element may be directly connectedwith the other element or intervening elements may also be present. Incontrast, when an element is referred to as being “directly connectedwith” another element, there are no intervening elements present.

A singular representation may include a plural representation unless itrepresents a definitely different meaning from the context. Terms suchas “include” or “has” are used herein and should be understood that theyare intended to indicate an existence of several components, functionsor steps, disclosed in the specification, and it is also understood thatgreater or fewer components, functions, or steps may likewise beutilized.

FIG. 1 is a schematic diagram showing an example of a wirelesscommunication system using BLUETOOTH low energy technology according toan embodiment of the present disclosure.

The wireless communication system may include a terminal 100 and aplurality of target devices 200, 300, 400 and 500.

The terminal 100 and the plurality of target devices 200, 300, 400 and500 may perform BLUETOOTH communication using BLUETOOTH low energy (BLE)(hereinafter, referred to as BLE) technology.

BLE technology has a relatively smaller duty cycle as compared toBLUETOOTH basic rate/enhanced data rate (BR/EDR) technology, enableslow-cost production and significantly reduces power consumption througha low data transfer rate. Due to such reduction in power consumption,some devices utilizing BLE may operate on only a coin cell battery formore than one year.

In addition, BLE technology simplifies a procedure of connecting devicesand has a smaller packet size than BLUETOOTH BR/EDR technology.

In BLE technology, (1) the number of radio frequency (RF) channels is40, (2) a data transfer rate of 1 Mbps is supported, (3) topology has ascattemet structure, (4) latency is 3 ms, (5) maximum current is 15 mAor less, (6) output power is 10 mW (10 dBm) or less, and (7) the BLEtechnology is mainly used in applications such as mobile phones,watches, sports, health care, sensors, and device control.

Meanwhile, the terminal 100 may operate as a client device of theplurality of target devices. In addition, the terminal may berepresented by a master device, a master, a client, a member, a sensordevice, a sink device, a collector, a first device, a handsfree device,and the like.

In addition, the plurality of target devices 200, 300, 400 and 500 mayoperate as a server device of the terminal 100. In addition, theplurality of target devices may be represented by a data service device,a slave device, a slave, a server, a conductor, a host device, agateway, a sensing device, a monitoring device, a second device, anaudio gate (AG), and the like.

The plurality of target devices refers to devices for providing data tothe terminal through a response upon receiving a data request from theterminal, by receiving data from the terminal and directly performingcommunication with the terminal.

In addition, the plurality of target devices sends a notificationmessage and an indication message to the terminal in order to providedata information to the terminal. In addition, the plurality of targetdevices receives a confirmation message corresponding to the indicationmessage from the terminal after transmitting the indication message tothe terminal.

In addition, the plurality of target devices may provide datainformation to a user through a display unit or receive a request fromthe user through a user input interface, during a process oftransmitting and receiving notification, indication, and confirmationmessages to and from the client device.

In addition, the plurality of target devices may read data from a memoryunit and write new data in the memory unit, during a process oftransmitting and receiving a message to and from the terminal.

The terminal generally refers to a device for requesting datainformation and data transmission from the plurality of target devices.

The terminal receives data from the plurality of target devices throughthe notification message, the indication message, etc., and transmits aconfirmation message in response to the indication message uponreceiving the indication message from the plurality of target devices.

The terminal may provide information to the user through an output unitand receive user input through an input unit, during a process oftransmitting and receiving messages to and from the plurality of targetdevices.

In addition, the terminal may read data from a memory unit and write newdata in the memory unit, in a process of transmitting and receivingmessages to and from the plurality of target devices.

FIGS. 2A-2D are diagrams illustrating problems which may occur upon BLEconnection.

Referring to FIG. 2A, in one example a user selects a BLUETOOTH icon 210to enter a BLUETOOTH connection menu.

In the connection menu show in FIG. 2B, when the user presses a scanbutton 220, the terminal 100 performs scanning. When scanning ends, asshown in FIG. 2C, the terminal displays a list 230 of scannedconnectable devices.

The user may confirm and select a device to be connected from the list.In this case, the terminal may be connected to the selected device.

As shown in FIG. 2D, the terminal executes a dedicated application forcontrolling the connected device, and the user may control the connecteddevice using the control screen of the dedicated application.

In order to connect the terminal with another device, many operationsmay have to be performed.

In addition, as shown in FIG. 2C, since the list of scanned connectabledevices includes device names such as WMRRD11-NC1012P, it may bedifficult for the user to grasp which list item corresponds to whichdevice and to accurately determine which device is desired to becontrolled.

Furthermore, various dedicated applications capable of controllingvarious devices may need to be identified and installed since thededicated application for controlling the connected device is executedat the terminal.

FIG. 3 is a block diagram illustrating a terminal 100 according to anembodiment of the present disclosure.

The terminal 100 may include a wireless communication unit 110, an inputunit 120, an artificial intelligence unit 130, a sensing unit 140, anoutput unit 150, an interface unit 160, a memory 170, a controller 180,a power supply unit 190 and the like.

FIG. 3 illustrates the terminal having various components, but it isunderstood that implementing all of the illustrated components is not arequirement, and that greater or fewer components may alternatively beimplemented.

In more detail, the wireless communication unit 110 of those componentsmay typically include one or more modules which permit wirelesscommunication between the terminal 100 and a wireless communicationsystem, between the terminal 100 and another terminal 100, or betweenthe terminal 100 and an external server. Further, the wirelesscommunication unit 110 typically includes one or more modules whichconnect the terminal 100 to one or more networks.

The wireless communication unit 110 may include at least one of abroadcast reception module 111, a mobile communication module 112, awireless Internet module 113, a short-range communication module 114, alocation information module 115 and the like.

The input unit 120 may include a camera 121 or an image input unit forinputting an image signal, a microphone 122 or an audio input module forinputting an audio signal, or a user input unit 123 (for example, atouch key, a push key (or a mechanical key), etc.) for allowing a userto input information. Audio data or image data collected by the inputunit 120 may be analyzed and processed by a user's control command.

An artificial intelligence unit 130 is responsible for processinginformation based on artificial intelligence technology and may includeone or more modules for performing at least one of learning ofinformation, inference of information, perception of information andprocessing of a natural language.

The artificial intelligence unit 130 may perform at least one oflearning, inference and processing of vast amounts of information (bigdata) such as information stored in the terminal, surroundingenvironmental information of the terminal and information stored in acommunicable external storage, using machine learning technology. Inaddition, the artificial intelligence unit 130 may control the terminalto predict (infer) executable operation of at least one terminal and toperform most feasible operation of the at least one predicted operation,using the information learned using the machine learning technology.

The machine learning technology refers to technology of collecting andlearning a large amount of information based on at least one algorithmand determining and predicting information based on the learnedinformation. Learning of information refers to operation of grasping thecharacteristics, rules and criteria of judgement of the information,quantifying a relationship between information and information, andpredicting new data using a quantified pattern.

An algorithm used by such machine learning technology may be astatistical based algorithm and may include, for example, a decisiontree using a tree structure as a prediction model, an artificial neuralnetwork for emulating the neural network structure and function of anorganism, genetic programing based on biological evolutionaryalgorithms, clustering for distributing observed examples into subsetssuch clusters, and a Monte-Carlo method of calculating the probabilityof a function value through a randomly extracted number.

As a field of machine learning technology, deep learning technologyrefers to technology of performing at least one of learning, determiningand processing of information using an artificial neural networkalgorithm. The artificial neural network may have a structure forconnecting a layer with a layer and transmitting data between the layerand the layer. Such deep learning technology may learn vast amounts ofinformation through an artificial neural network using a graphicprocessing unit (GPU) optimized for parallel computation.

Meanwhile, the artificial intelligence unit 130 may collect (sense,monitor, extract, detect or receive) signals, data, information, etc.input to or output from the components of the terminal in order tocollect vast amounts of information for applying machine learningtechnology. In addition, the artificial intelligence unit 130 maycollect (sense, monitor, extract, detect or receive) data, information,etc. stored in an external storage (e.g., a cloud server) connectedthrough communication. More specifically, collection of information maybe understood as the term including sensing of information through asensor, extroperation of information stored in the memory 170, orreception of information from the external storage throughcommunication.

The artificial intelligence unit 130 may sense information in theterminal, surrounding environment information of the terminal and userinformation through the sensing unit 140. In addition, the artificialintelligence unit 130 may receive a broadcast signal and/or broadcastrelated information, wireless signal, wireless data, etc. through thewireless communication unit 110. In addition, the artificialintelligence unit 130 may receive image information (or signal), audioinformation (or signal), data or information input by a user from theinput unit.

Such an artificial intelligence unit 130 may collect vast amounts ofinformation in real time in the background and learn the information,and store information processed in an appropriate form (e.g. knowledgegraph, command policy, personalization database, dialog engine, etc.) inthe memory 170.

In addition, when operation of the terminal is predicted based on theinformation learned using the machine learning technology, theartificial intelligence unit 130 may control the components of theterminal and send a control command for executing the predictedoperation to the controller 180, in order to execute the predictedoperation. The controller 180 may control the terminal based on thecontrol command to execute the predicted operation.

Meanwhile, when specific operation is performed, the artificialintelligence unit 130 may analyze history information indicatingperforming of the specific operation through machine learning technologyand update existing learned information based on the analyzedinformation. Therefore, the artificial intelligence unit 130 may improveinformation prediction accuracy.

Meanwhile, in this specification, the artificial intelligence unit 130and the controller 180 may be understood as the same component. In thiscase, the function performed by the controller 180 described in thisspecification may be described as being performed by the artificialintelligence unit 130, and the controller 180 may be referred to as theartificial intelligence unit 130 or the artificial intelligence unit 130may be referred to as the controller 180.

Alternatively, in this specification, the artificial intelligence unit130 and the controller 180 may be understood as different components. Inthis case, the artificial intelligence unit 130 and the controller 180may perform a variety of control on the terminal through data exchange.The controller 180 may perform at least one function on the terminalbased on a result derived from the artificial intelligence unit 130 orcontrol at least one of the components of the terminal. Further, theartificial intelligence unit 130 may operate under control of thecontroller 180. In some embodiments the artificial intelligence unit maybe implemented by one or more hardware controllers or processors,similar to that described with respect to the controller 180.

The sensing unit 140 may include one or more hardware sensors configuredto sense internal information of the terminal, the surroundingenvironment of the terminal, user information, and the like.

For example, the sensing unit 140 may include at least one of aproximity sensor 141, an illumination sensor 142, a touch sensor, anacceleration sensor, a magnetic sensor, a G-sensor, a gyroscope sensor,a motion sensor, an RGB sensor, an infrared (IR) sensor, a finger scansensor, a ultrasonic sensor, an optical sensor (for example, camera121), a microphone 122, a battery gauge, an environment sensor (forexample, a barometer, a hygrometer, a thermometer, a radiation detectionsensor, a thermal sensor, and a gas sensor), and a chemical sensor (forexample, an electronic nose, a health care sensor, a biometric sensor,and the like). The terminal 100 may be configured to utilize acombination of information sensed by at least two of the sensors.

The output unit 150 is typically configured to output various types ofinformation, such as audio, video, tactile output, and the like. Theoutput unit 150 may include at least one of a display unit 151, an audiooutput unit 152, a haptic module 153, and an optical output unit 154.The display unit 151 may have an inter-layered structure or anintegrated structure with a touch sensor in order to facilitate atouchscreen. The touchscreen may provide an output interface between theterminal 100 and a user, as well as function as the user input unit 123which provides an input interface between the terminal 100 and the user.

The interface unit 160 serves as an interface with various types ofexternal devices connected to the terminal 100. The interface unit 160,for example, may include at least one of wired or wireless ports,external power supply ports, wired or wireless data ports, memory cardports, ports for connecting a device having an identification module,audio input/output (I/O) ports, video I/O ports, earphone ports, and thelike. In some cases, the terminal 100 may perform assorted controlfunctions associated with a connected external device, in response tothe external device being connected to the interface unit 160.

The memory 170 is typically implemented to store data to support variousfunctions of the terminal 100. For instance, the memory 170 may beconfigured to store application programs executed in the terminal 100,data or instructions for operations of the terminal 100, data foroperation of the artificial intelligence unit 130 (e.g., at least onepiece of algorithm information for machine learning, etc.), and thelike. Some of these application programs may be downloaded from anexternal server via wireless communication. Other application programsmay be installed within the terminal 100 at time of manufacturing orshipping, for basic functions of the terminal 100 (for example,receiving a call, placing a call, receiving a message, sending amessage, and the like). The application programs may be stored in thefirst memory 170, installed in the terminal 100, and executed by thecontroller 180 to perform an operation (or function) of the terminal.

The controller 180 typically functions to control overall operation ofthe terminal 100, in addition to the operations associated with theapplication programs. The controller 180 may provide or processinformation or functions appropriate for a user by processing signals,data, information and the like, which are input or output by theabove-described components, or executing application programs stored inthe memory 170.

Furthermore, the controller 180 may control at least some of thecomponents illustrated in FIG. 3, in order to execute the applicationprograms stored in the memory 170. In addition, the controller 180 mayexecute the application programs by operating at least two of thecomponents included in the terminal 100 for operation.

The power supply unit 190 may receive external power or internal powerand supply appropriate power required for operating respectivecomponents included in the terminal 100 under control of the controller180. The power supply unit 190 may include a battery, and the batterymay be an embedded battery or a replaceable battery.

Hereinafter, each aforementioned component will be described in moredetail with reference to FIG. 3, prior to explaining various exemplaryembodiments implemented by the terminal 100 having the configuration.

First, the wireless communication unit 110 will be described. Thebroadcast reception module 111 of the wireless communication unit 110may receive a broadcast signal and/or broadcast associated informationfrom an external broadcast managing server via a broadcast channel. Thebroadcast channel may include a satellite channel and a terrestrialchannel. Two or more broadcast reception modules may be included in theterminal 100, for simultaneous reception of two or more broadcastchannels or for switching between broadcast channels.

The broadcast managing server may be a server which generates andtransmits a broadcast signal and/or broadcast associated information, ora server which receives a pre-generated broadcast signal and/orbroadcast associated information, and sends such items to the terminal.The broadcast signal may be implemented using any of a TV broadcastsignal, a radio broadcast signal, a data broadcast signal, and acombination of a data broadcast signal and a TV or radio broadcastsignal.

The broadcast signal may be encoded according to at least one of avariety of technical standards or broadcasting methods (for example,International Organization for Standardization (ISO), InternationalElectrotechnical Commission (IEC), Digital Video Broadcast (DVB),Advanced Television Systems Committee (ATSC), and the like) fortransmission and reception of digital broadcast signals. The broadcastreception module 111 may receive the digital broadcast signals using amethod appropriate for the transmission method utilized.

Examples of broadcast associated information may include informationassociated with a broadcast channel, a broadcast program, a broadcastservice provider, or the like. The broadcast associated information mayalso be provided via a mobile communication network, and in this case,received by the mobile communication module 112.

The broadcast associated information may be implemented in variousformats. For instance, broadcast associated information may include anElectronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB),an Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld(DVB-H), and the like. Broadcast signals and/or broadcast associatedinformation received via the broadcast reception module 111 may bestored in a suitable device, such as the memory 170.

The mobile communication module 112 may transmit and/or receive wirelesssignals to and from a base station, an external terminal, a server, andthe like over a mobile communication network, which is constructedaccording to technical standards or communication methods for mobilecommunications (for example, Global System for Mobile Communication(GSM), Code Division Multi Access (CDMA), CDMA2000 (Code Division MultiAccess 2000), EV-DO (Enhanced Voice-Data Optimized or EnhancedVoice-Data Only), Wideband CDMA (WCDMA), High Speed Downlink Packetaccess (HSDPA), HSUPA (High Speed Uplink Packet Access), Long TermEvolution (LTE), LTE-A (Long Term Evolution-Advanced), and the like).

Examples of wireless signals include audio call signals, video(telephony) call signals, or various formats of data to supportcommunication of text and multimedia messages.

The wireless Internet module 113 is configured to facilitate wirelessInternet access. This module may be internally or externally coupled tothe terminal 100. The wireless Internet module 113 may transmit and/orreceive wireless signals via communication networks according towireless Internet technologies.

Examples of such wireless Internet technology include Wireless LAN(WLAN), Wireless Fidelity (Wi-Fi), Wi-Fi Direct, Digital Living NetworkAlliance (DLNA), Wireless Broadband (WiBro), Worldwide Interoperabilityfor Microwave Access (WiMAX), High Speed Downlink Packet Access (HSDPA),HSUPA (High Speed Uplink Packet Access), Long Term Evolution (LTE),LTE-A (Long Term Evolution-Advanced), and the like. The wirelessInternet module 113 may transmit/receive data according to one or moreof such wireless Internet technologies, and other Internet technologiesas well.

In some embodiments, when the wireless Internet access is implementedaccording to, for example, WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE,LTE-A and the like, as part of a mobile communication network, thewireless Internet module 113 performs such wireless Internet access. Assuch, the wireless Internet module 113 may cooperate with, or functionas, the mobile communication module 112.

The short-range communication module 114 is configured to facilitateshort-range communication and to support short-range communication usingBLUETOOTH™, Radio Frequency IDentification (RFID), Infrared DataAssociation (IrDA), Ultra-WideBand (UWB), ZigBee, Near FieldCommunication (NFC), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, WirelessUSB (Wireless Universal Serial Bus), and the like. The short-rangecommunication module 114 in general supports wireless communicationbetween a terminal 100 and a wireless communication system, between theterminal 100 and another terminal 100 or between the terminal 100 and anetwork where another terminal 100 (or an external server) is located,via wireless area networks. One example of the wireless area networks isa wireless personal area networks.

In some embodiments, another terminal 100 may be a wearable device, forexample, a smart watch, smart glasses or a head mounted display (HMD),which is able to exchange data with the terminal 100 (or otherwisecooperate with the terminal 100). The short-range communication module114 may sense (or recognize) a wearable device capable of performingcommunication with the terminal 100 in the vicinity of the terminal 100.In addition, when the sensed wearable device is a device authenticatedto communicate with the terminal 100, the controller 180, for example,may transmit data processed in the terminal 100 to the wearable devicevia the short-range communication module 114. Hence, a user of thewearable device may use the data processed in the terminal 100 on thewearable device. For example, when a call is received in the terminal100, the user may answer the call using the wearable device. Also, whena message is received in the terminal 100, the user may check thereceived message using the wearable device.

The location information module 115 is generally configured to acquirethe position (or the current position) of the terminal. As an example,the location information module 115 includes a Global Position System(GPS) module, a Wi-Fi module, or both. For example, when the terminaluses a GPS module, the position of the terminal may be acquired using asignal sent from a GPS satellite.

As another example, when the terminal uses the Wi-Fi module, theposition of the terminal may be acquired based on information related toa wireless access point (AP) which transmits or receives a wirelesssignal to or from the Wi-Fi module. when desired, the locationinformation module 115 may alternatively or additionally function withany of the other modules of the wireless communication unit 110 toobtain data related to the position of the terminal. The locationinformation module 115 is used to acquire the position (or the currentposition) of the terminal and is not limited to a module for directlycalculating or acquiring the position of the terminal.

The input unit 120 may be configured to permit input of image or videoinformation (or signal), audio information (or signal), data or userinput information. For input of image or video information, the terminal100 may include one or a plurality of cameras 121. The camera 121 mayprocess image frames of still pictures or video obtained by imagesensors in a video or image capture mode. The processed image frames maybe displayed on the display unit 151 or stored in memory 170. In somecases, the plurality of cameras 121 provided in the terminal 100 may bearranged in a matrix configuration to permit a plurality of imageshaving various angles or focal points to be input to the terminal 100.As another example, the plurality of cameras 121 may be located in astereoscopic arrangement to acquire left and right images forimplementing a stereoscopic image.

The microphone 122 processes an external sound signal into an electricalaudio signal. The processed audio data may be processed in variousmanners according to a function (or an application program) beingexecuted in the terminal 100. when desired, the microphone 122 mayinclude assorted noise removing algorithms to remove unwanted noisegenerated in the course of receiving the external audio signal.

The user input unit 123 is a component that permits input by a user.When information is input through the user input unit 123, thecontroller 180 may control operation of the terminal 100 to correspondto the input information. The user input unit 123 may include one ormore of a mechanical input element (for example, a mechanical key, abutton located on a front and/or rear surface or a side surface of theterminal 100, a dome switch, a jog wheel, a jog switch, and the like),or a touch-sensitive input element, among others. As one example, thetouch-sensitive input element may be a virtual key or a soft key, whichis displayed on a touchscreen through software processing, or a touchkey which is located on the terminal at a location that is other thanthe touchscreen. On the other hand, the virtual key or the visual keymay be displayed on the touchscreen in various shapes, for example,graphic, text, icon, video, or a combination thereof.

The sensing unit 140 is generally configured to sense one or more ofinternal information of the terminal, surrounding environmentinformation of the terminal, user information, or the like and togenerate sensing signals corresponding thereto. The controller 180generally controls operation of the terminal 100 or execute dataprocessing, functions or operation associated with an applicationprogram installed in the terminal 100 based on such sensing signals. Thesensing unit 140 may be implemented using any of a variety of sensors,some of which will now be described in more detail.

The proximity sensor 141 may include a sensor to sense presence orabsence of an object approaching a detection surface, or an objectlocated near a detection surface, by using an electromagnetic field,infrared rays, or the like without a mechanical contact. The proximitysensor 141 may be arranged at an inner region of the terminal covered bythe touchscreen, or near the touchscreen.

The proximity sensor 141, for example, may include any of a transmissivetype photoelectric sensor, a direct reflective type photoelectricsensor, a mirror reflective type photoelectric sensor, a high-frequencyoscillation proximity sensor, a capacitance type proximity sensor, amagnetic type proximity sensor, an infrared-ray proximity sensor, andthe like. When the touchscreen is implemented as a capacitance type, theproximity sensor 141 may sense proximity of an object relative to thetouchscreen by changes of an electromagnetic field, which is responsiveto an approach of an object with conductivity. In this case, thetouchscreen (touch sensor) may also be categorized as a proximitysensor.

For convenience of description, the term “proximity touch” will often bereferred to herein to denote the scenario in which an object ispositioned to be proximate to the touchscreen without contacting thetouchscreen. The term “contact touch” will often be referred to hereinto denote the scenario in which an object makes physical contact withthe touchscreen. For the position corresponding to the proximity touchof the object relative to the touchscreen, such position will correspondto a position where the object is perpendicular to the touchscreen. Theproximity sensor 141 may sense proximity touch, and proximity touchpatterns (for example, distance, direction, speed, time, position,moving status, and the like).

In general, the controller 180 processes data (or information)corresponding to proximity touches and proximity touch patterns sensedby the proximity sensor 141, and cause output of visual informationcorresponding to the processed data on the touchscreen. In addition, thecontroller 180 may control the terminal 100 to execute differentoperations or process different data according to whether a touch withrespect to a point on the touchscreen is either a proximity touch or acontact touch.

A touch sensor may sense a touch applied to the touchscreen, such asdisplay unit 151, using any of a variety of touch methods. Examples ofsuch touch methods include a resistive type, a capacitive type, aninfrared type, an ultrasonic type, and a magnetic field type, amongothers.

As one example, the touch sensor may be configured to convert changes ofpressure applied to a specific part of the touchscreen, or convertcapacitance occurring at a specific part of the touchscreen, intoelectric input signals. The touch sensor may also be configured to sensenot only a touched position and a touched area, but also touch pressureand/or touch capacitance. A touch object is generally used to apply atouch input to the touch sensor. Examples of typical touch objectsinclude a finger, a touch pen, a stylus pen, a pointer, or the like.

When touch input is sensed by a touch sensor, corresponding signals maybe transmitted to a touch controller. The touch controller may processthe received signals, and then transmit corresponding data to thecontroller 180. Accordingly, the controller 180 may sense which regionof the display unit 151 has been touched. Here, the touch controller maybe a component separate from the controller 180, the controller 180, andcombinations thereof.

In some embodiments, the controller 180 may execute the same ordifferent controls according to a type of touch object that touches thetouchscreen or a touch key provided in addition to the touchscreen.Whether to execute the same or different control according to the objectwhich provides a touch input may be decided based on a current operatingstate of the terminal 100 or a currently executed application program,for example.

The touch sensor and the proximity sensor may be implementedindividually, or in combination, to sense various types of touches. Suchtouches include a short (or tap) touch, a long touch, a multi-touch, adrag touch, a flick touch, a pinch-in touch, a pinch-out touch, a swipetouch, a hovering touch, and the like.

When desired, an ultrasonic sensor may be implemented to recognizeposition information relating to a touch object using ultrasonic waves.The controller 180, for example, may calculate a position of a wavegeneration source based on information sensed by an optical sensor and aplurality of ultrasonic sensors. Since light is much faster thanultrasonic waves, the time for which the light reaches the opticalsensor is much shorter than the time for which the ultrasonic wavereaches the ultrasonic sensor. The position of the wave generationsource may be calculated using this fact. For instance, the position ofthe wave generation source may be calculated using the time differencefrom the time that the ultrasonic wave reaches the sensor based on thelight as a reference signal.

The camera 121 configuring the input unit 120 typically includes atleast one a camera sensor (CCD, CMOS etc.), a photo sensor (or imagesensors), and a laser sensor.

Implementing the camera 121 with a laser sensor may allow detection of atouch of a physical object with respect to a 3D stereoscopic image. Thephoto sensor may be laminated on the display device. The photo sensormay be configured to scan movement of the physical object in proximityto the touchscreen. In more detail, the photo sensor may include photodiodes and transistors at rows and columns to scan content received atthe photo sensor using an electrical signal which changes according tothe amount of light applied to the photo diode. Namely, the photo sensormay calculate the coordinates of the physical object according tovariation of light to thus obtain position information of the physicalobject.

The display unit 151 is generally configured to display (output)information processed in the terminal 100. For example, the display unit151 may display execution screen information of an application programexecuting at the terminal 100 or user interface (UI) and graphic userinterface (GUI) information in response to the execution screeninformation.

In some embodiments, the display unit 151 may be implemented as astereoscopic display unit for displaying stereoscopic images. A typicalstereoscopic display unit may employ a stereoscopic display scheme suchas a stereoscopic scheme (a glass scheme), an auto-stereoscopic scheme(glassless scheme), a projection scheme (holographic scheme), or thelike.

In general, a 3D stereoscopic image may include a left image (e.g., aleft eye image) and a right image (e.g., a right eye image). Accordingto how left and right images are combined into a 3D stereoscopic image,a 3D stereoscopic imaging method may be divided into a top-down methodin which left and right images are located up and down in a frame, anL-to-R (left-to-right or side by side) method in which left and rightimages are located left and right in a frame, a checker board method inwhich fragments of left and right images are located in a tile form, aninterlaced method in which left and right images are alternately locatedby columns or rows, and a time sequential (or frame by frame) method inwhich left and right images are alternately displayed on a time basis.

Also, as for a 3D thumbnail image, a left image thumbnail and a rightimage thumbnail may be generated from a left image and a right image ofan original image frame, respectively, and then combined to generate asingle 3D thumbnail image. In general, the term “thumbnail” may be usedto refer to a reduced image or a reduced still image. A generated leftimage thumbnail and right image thumbnail may be displayed with ahorizontal distance difference therebetween by a depth corresponding tothe disparity between the left image and the right image on the screen,thereby providing a stereoscopic space sense.

A left image and a right image required for implementing a 3Dstereoscopic image may be displayed on the stereoscopic display unitusing a stereoscopic processing unit. The stereoscopic processing unitmay receive the 3D image (the image of the reference view point and theimage of the extended view point) and extract the left image and theright image, or may receive the 2D image and change it into a left imageand a right image.

The audio output unit 152 is generally configured to output audio data.Such audio data may be obtained from any of a number of differentsources, such that the audio data may be received from the wirelesscommunication unit 110 or may have been stored in the memory 170. Theaudio data may be output in modes such as a signal reception mode, acall mode, a record mode, a voice recognition mode, a broadcastreception mode, and the like. The audio output unit 152 may output anaudio signal related to a particular function (e.g., a call signalreception sound, a message reception sound, etc.) performed by theterminal 100. The audio output unit 152 may also be implemented as areceiver, a speaker, a buzzer, or the like.

A haptic module 153 may be configured to generate various tactileeffects that a user feels. A typical example of a tactile effectgenerated by the haptic module 153 is vibration. The strength, patternand the like of the vibration generated by the haptic module 153 may becontrolled by user selection or setting by the controller. For example,the haptic module 153 may output different vibrations in a combiningmanner or a sequential manner.

Besides vibration, the haptic module 153 may generate various othertactile effects, including an effect by stimulation such as a pinarrangement vertically moving to contact skin, a spray force or suctionforce of air through a jet orifice or a suction opening, a touch to theskin, a contact of an electrode, electrostatic force, an effect byreproducing the sense of cold and warmth using an element that mayabsorb or generate heat, and the like.

The haptic module 153 may also be implemented to allow the user to feela tactile effect through a muscle sensation such as the user's fingersor arm, as well as transferring the tactile effect through directcontact. Two or more haptic modules 153 may be provided according to theparticular configuration of the terminal 100.

An optical output unit 154 may output a signal for indicating eventgeneration using light of a light source. Examples of events generatedin the terminal 100 may include message reception, call signalreception, a missed call, an alarm, a schedule notice, an emailreception, information reception through an application, and the like.

A signal output by the optical output unit 154 may be implemented insuch a manner that the terminal emits monochromatic light or light witha plurality of colors. The signal output may be terminated as theterminal senses that a user has checked the generated event, forexample.

The interface unit 160 serves as an interface for external devices to beconnected with the terminal 100. For example, the interface unit 160 mayreceive data transmitted from an external device, receive and transferpower to elements and components within the terminal 100, or transmitinternal data of the terminal 100 to such external device. The interfaceunit 160 may include wired or wireless headset ports, external powersupply ports, wired or wireless data ports, memory card ports, ports forconnecting a device having an identification module, audio input/output(I/O) ports, video I/O ports, earphone ports, or the like.

The identification module may be a chip that stores various informationfor authenticating authority of using the terminal 100 and may include auser identity module (UIM), a subscriber identity module (SIM), auniversal subscriber identity module (USIM), and the like. In addition,the device having the identification module (also referred to herein asan “identifying device”) may take the form of a smart card. Accordingly,the identifying device may be connected with the terminal 100 via theinterface unit 160.

When the terminal 100 is connected with an external cradle, theinterface unit 160 may serve as a passage to allow power from the cradleto be supplied to the terminal 100 or may serve as a passage to allowvarious command signals input by the user from the cradle to betransferred to the terminal 100 therethrough. Various command signals orpower input from the cradle may operate as signals for recognizing thatthe terminal 100 is properly mounted on the cradle.

The memory 170 may store programs to support operations of thecontroller 180 and store input/output data (for example, phonebook,messages, still images, videos, etc.). The memory 170 may store datarelated to various patterns of vibrations and audio which are output inresponse to touch inputs on the touchscreen.

The memory 170 may include one or more types of storage mediumsincluding a flash memory type, a hard disk type, a solid state disk(SSD) type, a silicon disk drive (SDD) type, a multimedia card microtype, a card-type memory (e.g., SD or DX memory, etc.), a Random AccessMemory (RAM), a Static Random Access Memory (SRAM), a Read-Only Memory(ROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM),a Programmable Read-Only memory (PROM), a magnetic memory, a magneticdisk, an optical disk, and the like. The terminal 100 may also beoperated in relation to a network storage device that performs thestorage function of the memory 170 over a network, such as the Internet.

As described above, the controller 180 may typically control the generaloperations of the terminal 100. For example, the controller 180 may setor release a lock state for restricting a user from inputting a controlcommand with respect to applications when a status of the terminal meetsa preset condition.

The controller 180 may also perform the controlling and processingassociated with voice calls, data communications, video calls, and thelike, or perform pattern recognition processing to recognize ahandwriting input or a picture drawing input performed on thetouchscreen as characters or images, respectively. In addition, thecontroller 180 may control one or a combination of those components inorder to implement various exemplary embodiments disclosed herein.

The power supply unit 190 receives external power or provide internalpower and supply the appropriate power required for operating respectiveelements and components under control of the controller 180. The powersupply unit 190 may include a battery, which is typically rechargeableor be detachably coupled to the terminal body for charging.

The power supply unit 190 may include a connection port. The connectionport may be configured as one example of the interface unit 160 to whichan external charger for supplying power to recharge the battery iselectrically connected.

As another example, the power supply unit 190 may be configured torecharge the battery in a wireless manner without use of the connectionport. In this example, the power supply unit 190 may receive power,transferred from an external wireless power transmitter, using at leastone of an inductive coupling method which is based on magnetic inductionor a magnetic resonance coupling method which is based onelectromagnetic resonance.

Various embodiments described herein may be implemented in acomputer-readable medium or a similar medium thereof using, for example,software, hardware, or any combination thereof.

Meanwhile, although the input unit 120 and the sensing unit 140 aredescribed as separate components, the sensing unit 140 may include theinput unit 120.

In this specification, the term “wireless communication unit 110” may beused interchangeably with the term “communication unit”.

In this specification, the term “memory 170” may be used interchangeablywith the term “storage unit”.

In this specification, the term “controller 180” may be usedinterchangeably with the term “processor”.

In this specification, the term “wireless communication unit” may beused interchangeably with the term “wireless communicator”.

In this specification, the term “input unit” may be used interchangeablywith the term “input interface”.

In this specification, the term “user input unit” may be usedinterchangeably with the term “user input interface”.

In this specification, the term “sensing unit” may be usedinterchangeably with the term “sensor”.

In this specification, the term “output unit” may be usedinterchangeably with the term “output interface”.

In this specification, the term “display unit” may be usedinterchangeably with the term “display”.

In this specification, the term “audio output unit” may be usedinterchangeably with the term “audio output interface”.

Meanwhile, the controller 180 may include an application-specificintegrated circuit (ASIC), another chipset, a logic circuit and/or adata processor.

In this specification, the term “artificial intelligence unit 130” maybe used interchangeably with the term “controller”, “processor”,“artificial intelligence controller”, or “artificial intelligenceprocessor”.

Meanwhile, the wireless communication unit 110 may include a networkinterface.

The network interface refers to a device for enabling a terminal toperform wired or wireless communication with a plurality of targetdevices and may include an energy efficient interface and a legacyinterface.

The energy efficient interface refers to a device for performinglow-power wireless communication with low energy consumption and means aunit (or a module) for enabling a terminal to discovering a targetdevice to be connected or to transmit data.

The legacy interface refers to a device for wireless communication andmeans a unit (or a module) for enabling a terminal to discoveringanother device to be connected or to transmit data.

Meanwhile, the display unit 151 may be a unit (or a module) foroutputting data received via the network interface or data stored in thememory 170 under control of the controller 180.

The controller 180 controls the network interface to receiveadvertisement information from a target device, controls a communicationunit to transmit a scan request to the target device and to receive ascan response from the target device in response to the scan request,and controls the network interface to transmit a connect request messageto the server device for BLUETOOTH connection establishment with thetarget device.

In addition, the controller 180 controls the communication unit to readand write data from and in the target device using an attributeprotocol, after a BLUETOOTH LE connection is established through theabove-described connection procedure.

Hereinafter, procedures of BLUETOOTH low energy (BLE) technology will bebriefly described.

BLE procedures may be divided into a device filtering procedure, anadvertising procedure, a scanning procedure, a discovering procedure, aconnecting procedure, etc.

Device Filtering Procedure

The device filtering procedure is a method of reducing the number ofdevices for responding to requests, instructions, notification, etc. ina controller stack.

Since it is unnecessary for all devices to respond to received requests,the controller stack may reduce the number of transmitted requests,thereby reducing power consumption in the BLE controller stack.

An advertisement device or a scanning device may perform the devicefiltering procedure in order to restrict devices for receiving anadvertisement packet, a scan request or a connect request.

Here, the advertisement device refers to a device for transmitting anadvertisement event, that is, performing advertisement, and is alsoreferred to as an advertiser. A plurality of target devices 200, 300,400 and 500 corresponds to the advertisement device.

The scanning device refers to a device for performing scanning or adevice for transmitting a scan request. The terminal 100 corresponds tothe scanning device.

In BLE, when the scanning device receives some advertisement packetsfrom the advertisement device, the scanning device transmits the scanrequest to the advertisement device.

However, when the device filtering procedure is used and thus scanrequest transmission is unnecessary, the scanning device may ignoreadvertisement packets transmitted from the advertisement device.

Even in a connect request process, the device filtering procedure may beused. when the device filtering procedure is used in the connect requestprocess, by ignoring a connect request, it is unnecessary to transmit aresponse to the connect request.

Advertising Procedure

The advertisement device performs an advertising procedure in order toperform a nondirectional broadcast to devices in an area.

Here, the nondirectional broadcast refers to a broadcast in alldirections, rather than a broadcast in a specific direction.

In contrast, a directional broadcast refers to a broadcast in a specificdirection. The nondirectional broadcast is performed without aconnecting procedure between the advertisement device and a device in alistening state (hereinafter, referred to as a listening device).

The advertising procedure is used to establish a BLUETOOTH connectionwith a neighboring initiating device.

Alternatively, the advertising procedure may be used to provide aperiodic broadcast of user data to scanning devices which are listeningthrough an advertisement channel.

In the advertising procedure, all advertisements (or advertisementevents) are broadcast through an advertisement physical channel.

The advertisement devices may receive a scan request from the listeningdevices which are listening in order to obtain additional user data fromthe advertisement device. The advertisement device transmits a responseto the scan request to the device, which has transmitted the scanrequest, through the same advertisement physical channel as theadvertisement physical channel, through which the scan request has beenreceived.

The broadcast user data sent as a portion of the advertisement packetsis dynamic data, but the scan response data is static data.

The advertisement device may receive the connect request from theinitiating device through the advertisement (broadcast) physicalchannel. when the advertisement device uses a connectable advertisementevent and the initiating device is not filtered by the device filteringprocedure, the advertisement device stops advertisement and enters aconnected mode. The advertisement device may resume advertisement afterthe connected mode.

Scanning Procedure

A device for performing scanning, that is, a scanning device, performs ascanning procedure in order to listen to a nondirectional broadcast ofuser data from the advertisement devices using the advertisementphysical channel.

The scanning device transmits the scan request to the advertisementdevice through the advertisement physical channel in order to requestadditional data from the advertisement device. The advertisement devicetransmits a scan response including additional data requested by thescanning device through the advertisement physical channel in responseto the scan request.

The scanning procedure may be used while a connection with another BLEdevice is established over a BLE piconet.

When the scanning device is in an initiator mode in which the broadcastadvertisement event may be received and a connect request may beinitiated, the scanning device may transmit the connect request to theadvertisement device through the advertisement physical channel, therebystarting a BLUETOOTH connection with the advertisement device.

When the scanning device transmits the connect request to theadvertisement device, the scanning device stops initiator mode scanningfor additional broadcast and enters the connected mode.

Discovering Procedure

Devices capable of performing BLUETOOTH communication (hereinafter,referred to as “BLUETOOTH devices”) perform the advertising procedureand the scanning procedure in order to discover neighboring devices orto be discovered by other devices in a given area.

The discovering procedure is performed asymmetrically. A BLUETOOTHdevice which is trying to discover other neighboring devices is referredto as a discovering device and listens in order to discover devices foradvertising a scannable advertisement event. A BLUETOOTH device, whichmay be discovered by and used in other devices, is referred to as adiscoverable device, and actively broadcasts an advertisement eventthrough the advertisement (broadcast) physical channel in order to bescannable by other devices.

The discovering device and the discoverable device may already beconnected to other BLUETOOTH devices over the piconet.

Connecting Procedure

The connecting procedure is asymmetrical. The connecting procedure makesa request such that, while a specific BLUETOOTH device performs theadvertising procedure, another BLUETOOTH device performs the scanningprocedure.

That is, the advertising procedure may be targeted and, as a result,only one device responds to the advertisement. After the accessibleadvertisement event is received from the advertisement device, theconnect request may be transmitted to the advertisement device throughthe advertisement (broadcast) physical channel, thereby initiating aconnection.

Next, the operation states of the BLE technology, that is, anadvertising state, a scanning state, an initiating state and aconnection state, will be briefly described.

Advertising State

A link layer (LL) enters an advertising state by an indication of a host(stack). when the link layer is in an advertising state, the link layertransmits advertisement packet data units (PDUs) in the advertisementevents.

Each advertisement event includes at least one advertisement PDU and theadvertisement PDUs are transmitted through used advertisement channelindices. The advertisement event may be terminated when theadvertisement PDUs have been transmitted through the used advertisementindices or may be terminated earlier when the advertisement device needsto secure a space in order to perform other functions.

Scanning State

The link layer enters a scanning state by an indication of the host(stack). In the scanning state, the link layer listens to advertisementchannel indices.

The scanning state includes a passive scanning type and an activescanning type, and each scanning type is determined by the host.

A separate time for performing scanning or an advertisement channelindex is not defined.

In the scanning state, the link layer listens to the advertisementchannel index during a scanWindow duration. A scanInterval is defined asan interval between start points of two consecutive scan windows.

When scheduling collision does not occur, the link layer listens forcompletion of all scanIntervals of the scanWindow as indicated by thehost. In each scanWindow, the link layer scans other advertisementchannel indices. The link layer uses all available advertisement channelindices.

In passive scanning, the link layer only receives packets and does nottransmit any packets.

In active scanning, the link layer listens in order to depend on theadvertisement PDU type capable of requesting additional informationrelated to the advertisement device and the advertisement PDUs from theadvertisement device.

Initiating State

The link layer enters the initiating state by an indication by the host(stack).

When the link layer is in the initiating state, the link layer listensto the advertisement channel indices.

During the initiating state, the link layer listens to the advertisementchannel index during the scanWindow duration.

Connection State

The link layer enters the connection state when the device fortransmitting a connection request, that is, the initiating device,transmits a CONNECT_REQ PDU to the advertisement device or when theadvertisement device receives CONNECT_REQ PDU from the initiatingdevice.

After the link layer enters the connection state, a connection isconsidered to be established. However, the connection may not beestablished when the link layer enters the connection state. Adifference between a newly established connection and a previouslyestablished connection is only a link layer connection supervisiontimeout.

When two devices are connected, the two devices may perform differentfunctions.

A link layer for performing a master function is called a master and alink layer for performing a slave function is called a slave. The mastercontrols timing of a connection event, and the connection event refersto a point in time when the master is synchronized with the slave.

FIG. 4 is a diagram illustrating a method of operating a wirelesscommunication system according to an embodiment of the presentdisclosure.

The plurality of target devices 200 and 300 may transmit advertisementinformation to the terminal 100 (S410 and S420).

In addition, the terminal 100 may scan neighboring target devices(S430).

This will be described with reference to FIGS. 5 and 6.

For example, the first target device 200 will be described. The firsttarget device 200 in the advertising state may transmit advertisementinformation to neighboring devices for each advertisement event.

Here, a time between advertisement events may be defined as anadvertising interval. In addition, the first target device 200 maytransmit advertisement information to neighboring devices at eachadvertising interval.

The advertisement information may be used interchangeably with the term“advertisement packet” or “advertisement message”.

The first target device 200 may transmit an advertisement packetincluding specific information to neighboring devices through aprocedure of setting the advertisement packet.

Specifically, the first target device 200 performs an advertisementparameter setting procedure in order to include the specific informationin the advertisement packet in a standby state.

First, the host of the first target device 200 transmits a SetAdvertising Parameter Command to a controller in order to set parametersrelated to an advertisement packet transmission interval and an address.

The maximum transmission interval and minimum transmission interval ofthe advertisement packet, the type of the advertisement packet, the typeof the address used in the advertisement packet, the advertisementchannel used to transmit the advertisement packet, etc. may be setthrough the Set Advertising Parameter Command.

In addition, the host transmits a Set Advertising Data Command to thecontroller in order to set data used in advertisement packets includinga data field.

The first target device may set data included in the advertisementpacket through the Set Advertising Data Command.

For example, in order for the first target device to include data for atransport discovery service in the advertisement packet, data forTabular Data Stream (TDS) may be included in the advertisement packetthrough the Set Advertising Data Command.

Thereafter, the host transmits an Enable Advertisement Command to thecontroller in order to start or stop operation of transmitting theadvertisement packet.

The controller receives the Enable Advertisement Command from the hostand then continuously transmits the advertisement packet to neighboringdevices until the Enable Advertisement Command for stopping transmissionof the advertisement packet is received from the host.

Thereafter, the first target device 200 enters the advertising state fortransmitting the advertisement packet, and the controller of the firsttarget device 200 periodically transmits the advertisement packet in theadvertisement event.

The terminal 100 may set scanning parameters to perform scanningoperation, in order to receive the advertisement packet from theneighboring devices in the standby state.

First, the host of the terminal 100 transmits a Set Scan ParametersCommand to the controller in order to set parameters for scanning.

The type of the scanning operation such as passive scanning or activescanning, an interval at which scanning operation is performed, ascanWindow and the type of an address used in scan request packets maybe set through the Set Scan Parameters Command.

After parameters for scanning operation is set through the Set ScanParameters Command, the host transmits an Enable Scanning Command to thecontroller in order to start scanning operation.

Thereafter, the terminal 100 enters the scanning state in order toperform scanning operation and receives advertisement packets from theneighboring devices during the scanning window duration.

For example, the advertisement packets transmitted from the plurality oftarget devices may be received during the scanning duration of theterminal 100.

When the controller of Device 1 receives the advertisement packet, anadvertising report is generated and transmitted to the host in order toreport the advertisement packet.

At this time, the advertisement report may be calledLE_Advertising_Report_Event, and may be generated based on one or aplurality of advertisement packets.

FIG. 6 is a diagram showing an example of the data format anadvertisement packet.

Referring to FIG. 6, the advertisement packet has an Advertising Data(AD) structure, and the terminal 100 may parse the advertisement packetin order to find desired data.

Specifically, the plurality of target devices may include specific datain the advertisement packet in the AD structure through theabove-described advertisement parameter setting procedure.

For example, the plurality of target devices may configure theadvertisement packet in order to transmit transmission (Tx) power, datafor providing TDS, and manufacturer information, which is informationrelated to a manufacturer, through the advertisement packet.

The advertisement packet may be composed of a plurality of ADstructures, and each AD structure may be composed of a length type value(LTV) structure.

A Length field may indicate the length of data, an AD Type field mayindicate the type of included data, and AD Data may include actual data.

That is, each AD structure may include the Length field, the AD Typefield and the AD Data field.

Here, the AD type may indicate the type of the data included in AD Data.In addition, the AD Data field may include data according to AD Type,and may be composed of one or a plurality of LTV structures.

Meanwhile, the advertisement information transmitted by the targetdevice may include identification information of the target device. Inaddition, the identification information of the target device may be themodel ID of the target device.

For example, the advertisement information transmitted by the secondtarget device 300 may be the unique model ID of the second target device300 which is the identification information of the second target device300.

In another example, the advertisement information transmitted by thefirst target device 200 may be the unique model ID of the first targetdevice 200 which is the identification information of the first targetdevice 200.

In this case, the plurality of target devices may include theidentification information of each of the plurality of target devices inthe advertisement packet through the above-described advertisementparameter setting procedure.

In this case, the advertisement packet may include data indicating themodel ID of the target device. Specifically, an extended field includingdata indicating the model ID of the target device may be configuredusing Manufacturer AD type. In this case, the extended field may includean AI camera field such that the terminal 100 recognizes that the targetdevice is recognized through photographing, and may include a data field(Model ID for AI camera) indicating the model ID of the target device.In this case, data provided by the manufacturer of the target device(e.g., the ID of the manufacturer (Company ID), detailed data providedby the manufacturer, etc.) may also be included.

In addition, the advertisement information may include the accessaddress of each of the plurality of target devices for accessing theplurality of target devices.

Meanwhile, the plurality of target devices may include a plurality ofnetwork interfaces (e.g., Wi-Fi, WiGig, BR/EDR, etc.) in addition toBLUETOOTH LE. In this case, the advertisement message may includeinformation on the network interface supported by the target device.

Meanwhile, the controller 180 may receive a plurality of pieces ofadvertisement information including the identification information ofthe plurality of target devices from the plurality of target devices.

Specifically, the terminal 100 is initially in the standby state. In thestandby state, the terminal 100 may not receive a message.

In addition, in the standby state, the terminal 100 may enter theadvertising sate, the scanning state or the initiating state.

In order to scan a neighboring target device supporting BLUETOOTH in thestandby state, the terminal 100 may perform scanning. In this case, theterminal 100 may enter the scanning state and receive the advertisementpacket from the neighboring target device.

In this case, the controller 180 may parse the received advertisementpacket and acquire the model ID and access address of the target device.

For example, the controller 180 may parse the advertisement packetreceived from the first target device 200 and acquire the identificationinformation of the first target device 200 (the model ID of the firsttarget device 200) and the access address.

In another example, the controller 180 may parse the advertisementpacket received from the second target device 300 and acquire theidentification information of the second target device 300 (the model IDof the second target device 300) and the access address.

Meanwhile, the terminal 100 may acquire information on an alternativecommunication element (e.g., BLUETOOTH BR/EDR, Wi-Fi, Wi-Fi Direct,etc.) supported by the plurality of target devices through BLUETOOTH LEand information on a service (e.g., BLUETOOTH BR/EDR A2DP HFP, Wi-FiDirect Miracast, Wi-Fi Direct File Transfer, etc.) capable of beingprovided through the alternative communication element.

The terminal 100 may be connected to the target device through thealternative communication element based on the acquired information andmay provide various services through the connected alternativecommunication element.

Specifically, the terminal and the plurality of target devices mayexchange information on the alternative communication element supportedby each device and information on the service provided through thealternative communication element in the connection procedure ofBLUETOOTH LE.

More specifically, the terminal 100 is in the scanning state in order toscan neighboring devices supporting BLUETOOTH, and the plurality oftarget devices is in the advertising state.

The plurality of target devices transmits advertisement information tothe terminal 100 in the advertising state. At this time, theadvertisement information may be transmitted to a plurality of devicesin a broadcast manner through the advertisement channel, as describedabove.

Meanwhile, the terminal 100 may receive the advertisement informationduring the scanning window duration in the scanning state.

At this time, the advertisement information may include alternativecommunication element information indicating information on thealternative communication element supported by each of the plurality ofdevices and service information indicating information on the serviceprovided through the alternative communication element.

The service information may be included as list information in which aplurality of services is listed. The alternative communication elementinformation and the service information may be basic information forallowing the terminal 100 to select the alternative communicationelement and service to be enabled through GATT of BLUETOOTH LE.

The terminal 100, which has acquired the information on the alternativecommunication element supported by the target device and the informationon the service provided through the alternative communication elementthrough the advertisement information, may transmit a scan requestmessage to the target device, when additional information is necessary.

Thereafter, the terminal 100 may receive a scan response messageincluding the additional information from the target device.

In addition, the additional information may include the local name ofthe target device, a device class, a device type and/or a major serviceclass.

Returning to FIG. 4, the controller 180 may control the camera tophotograph the first target device among the plurality of target devices(S440). In addition, the controller 180 may acquire identificationinformation of the first target device using the photographed image ofthe first target device (S450).

This will be described with reference to FIGS. 7 and 8.

The controller 180 may acquire the identification information of thefirst target device using a learned machine learning model. The processof learning the machine learning model is shown in FIG. 7.

Referring to FIG. 7, data for training of a neural network may becollected (S710). Here, the data is related to the plurality of targetdevices and may include an image, model ID, description, product SKUidentification numbers, or the like, of each of the plurality of targetdevices.

Meanwhile, the collected data may be preprocessed (S720). Specifically,the collected data may be processed and classified into a state suitablefor machine learning. In this case, an image format supported by alearning program, the number of data, data of each model, etc. may beclassified/collected. In addition, the collected data may be dividedinto training data and review data.

The trained machine learning model may be generated by training theneural network using the training data in a machine learning manner(S730). Meanwhile, as the learning algorithm, Naive-Bayes, K-NearestNeighbors, Logistic Regression, Support Vector Machine, etc. may be usedin addition to the neural network.

The learning process of the neural network may be a process of labelingthe image of the target device with identification information (modelID) and performing training.

In order to improve a recognition rate, the images of the target devicephotographed at various angles and distances with various intensities ofillumination may be used as training data, and images obtained byphotographing a portion or the whole of the target device may be used astraining data.

The learning process of the neural network may be a process of labelingthe image of each of the plurality of target devices with identificationinformation (model ID) and performing training.

The machine learning model generated by training may be a model trainedby labeling each of the plurality of target devices with identificationinformation. when an image including a target device is input, thetrained machine learning model may estimate and output theidentification information (model ID), with which the target device islabeled.

When training ends, the result of training using the review data, thatis, the trained machine learning model, is confirmed (S740), and, whenaccuracy of the result of review exceeds a target value, the trainedmachine learning model may be installed in the terminal 100 (S750).

In this case, a program for implementing the machine learning model maybe stored in the storage unit.

In one embodiment of the present disclosure, as shown in FIG. 8, thecontroller 180 may control the camera of the terminal to photograph afirst target device 600.

Specifically, when an input of selecting an icon for a BLUETOOTH LowEnergy (BLE) connection is received, the controller 180 may enable thephotographing function of the camera.

When the photographing function of the camera is enabled, the controller180 may display a preview screen 810 indicating a preview image receivedthrough the camera.

In this case, the user may capture an image of the first target device600 to be controlled of the plurality of target devices. when the targetdevice is included in the image received through the camera, thecontroller 180 may perform photographing to acquire the photographedimage of the first target device.

When the target device is included in the image received through thecamera, the controller 180 may perform photographing at a predeterminedtime interval to acquire the photographed image of the first targetdevice.

FIG. 9 is a diagram illustrating a method of generating a list ofidentification information estimated using a photographed imageaccording to an embodiment of the present disclosure.

The controller 180 may photograph the first target device (S810). Inaddition, when the photographed image of the first target device isacquired, the controller 180 may estimate the identification informationof the first target device (S820).

Specifically, the controller 180 may input the photographed image of thefirst target device to the trained machine learning model. In this case,the trained machine learning model may estimate the identificationinformation of the first target device using the input image and outputthe estimated result, as shown in FIG. 9 b.

Meanwhile, the trained machine learning model may estimate a pluralityof pieces of identification information. when the shapes of the firsttarget device and the second target devices are equal (the same producthaving the same appearance and different versions) or when the shapes ofthe first target device and the second target device are similar (seriesproducts having similar appearances), the trained machine learning modelmay estimate the plurality of pieces of identification information ofthe photographed first target device and output the estimated result.

When the plurality of pieces of identification information is estimatedand output, the controller 180 may generate a list of estimatedidentification information (S830). In some embodiments, multiple trainedmachine learning models may be implemented, where each learning model isdedicated to being trained and outputting a different piece ofidentification information. For the purposes of this discussion, onelearning model may be discussed, however it will be understood thatvarious embodiments may include implementation of multiple dedicatedlearning models for generating different data points.

Returning to FIG. 4, the controller 180 may select a target device to beconnected to the terminal 100 from among the plurality of scanned targetdevices (S460).

Specifically, the controller 180 may select the first target device asthe target device to be connected to the terminal among the plurality oftarget devices, using the photographed image of the first target deviceand the plurality of pieces of advertisement information received fromthe plurality of target devices.

This will be described with reference to FIGS. 10 and 11.

FIGS. 10 and 11 are diagrams illustrating a method of selecting a targetdevice for connection and control using estimated identificationinformation and identification information of a plurality of targetdevices acquired by scanning, according to an embodiment of the presentdisclosure.

Assume that the terminal 100 receives the advertisement information ofthe first target device, the advertisement information of the secondtarget device, the advertisement information of the third target deviceby scanning the first target device, the second target device and thethird target device.

In addition, assume that the third target device does not support atarget device recognition service through photographing and theadvertisement information transmitted by the third target device doesnot include identification information (model ID).

In addition, assume that the first target device is photographed, thephotographed image of the first target device is input to the machinelearning model, and the machine learning model estimates the targetdevice included in the image as the first target device and outputs theidentification information of the first target device.

As shown in FIG. 11a , the controller 180 may input the photographedimage to the machine learning model and estimate the identificationinformation of the target device included in the photographed image(S1010). In this case, since the first target device is photographed,the machine learning model may determine that the target device includedin the photographed image is the first target device, and output theidentification information of the first target device.

Meanwhile, when the plurality of target devices is scanned and theplurality of pieces of advertisement information is received, thecontroller 180 may generate a list of the plurality of scanned targetdevices using the plurality of pieces of advertisement information(S1020).

Meanwhile, in the case of the target device which does not support thetarget device recognition service using photographing, the advertisementinformation may not include identification information (model ID).

Accordingly, the controller 180 may generate a list of target devices,which have transmitted the advertisement information including theidentification information (model ID). That is, as shown in FIG. 11b , alist of target devices including the identification information of thefirst target device (model ID) and identification information of thesecond target device (model ID) may be generated.

Meanwhile, the controller 180 may select a target device to be connectedwith the terminal 100 from among the plurality of target devices.

Specifically, the controller 180 may select a target device to beconnected with the terminal 100 from among the plurality of targetdevices, using the estimated identification information and theidentification information included in the plurality of pieces ofadvertisement information.

For example, the controller 180 may find the estimated identificationinformation in the scanned device list (S1030).

Meanwhile, since the machine learning model estimates the identificationinformation of the target device included in the photographed image asthe identification information of the first target device and the listof the scanned target devices includes the identification information ofthe first target device, as shown in FIG. 11c , the controller 180 mayselect the first target device as a target device to be connected.

Returning to FIG. 4, the controller 180 may enable a connection with theselected first target device (S470).

Specifically, the controller 180 may control the network interface totransmit the connect request to the first target device in order toestablish a connection with the first target device using the BLUETOOTHLE method. In this case, the access address included in theadvertisement information transmitted by the first target device may beused.

Meanwhile, when the connect request is received from the terminal 100,the first target device may enter the connected mode to establish aBLUETOOTH LE connection with the terminal 100.

Meanwhile, the controller 180 may control the network interface to reador write data from or in the target device using the attribute protocol,after a BLUETOOTH LE connection is established through the connectionprocedure.

Recently, technologies for identifying a subject using an AI camera haveappeared. However, these technologies may only identify the type of thesubject (a washing machine, a refrigerator, etc.), but may notaccurately identify the model of the target device.

However, in the present disclosure, the target device is estimated usingmachine learning and a target device to be connected is using theidentification information included in the advertisement packet. Whenthe user captures the target device to be connected using the camera,the photographed target device may be accurately identified to establisha connection.

For example, when there are several electronic devices having differentmodels and similar appearances, accuracy of identification throughestimation of the target device using machine learning may besignificantly lowered. However, in the present disclosure, thephotographed target device may be accurately recognized by combining thephotographed information with the model ID included in the advertisementpacket, thereby establishing a connection.

In addition, in embodiments of the present disclosure, the traditionallyrequired operations of performing scanning by user operation of pressingthe scanning button, displaying a list of scanned devices, and selectinga device to be connected by the user may be omitted. In addition, aprocess of displaying the list of scanned devices and selecting a modelname from the list in a state in which the user knows the model names ofthe target devices may be omitted.

That is, instead of the existing complex and difficult method, aconnection may be established by only capturing a device to be connectedusing a camera, thereby improving user convenience.

Meanwhile, the controller 180 may enable a connection with the firsttarget device using a communication method corresponding to a serviceprovided by the first target device.

Specifically, when the service provided by the first target device isenabled by a BLUETOOTH LE method, the controller 180 may perform theservice using a previously established BLUETOOTH LE connection.

In contrast, when the service provided by the first target device isenabled by a communication method other than the BLUETOOTH LE method,the controller 180 may perform handover to an alternative communicationelement in order to perform the service.

As described above, the terminal 100 may acquire information on analternative communication element (e.g., BLUETOOTH BR/EDR, Wi-Fi, Wi-FiDirect, etc.) supported by the plurality of target devices throughBLUETOOTH LE and information on a service (e.g., BLUETOOTH BR/EDR A2DPHFP, Wi-Fi Direct Miracast, Wi-Fi Direct File Transfer, etc.) capable ofbeing provided through the alternative communication element.

In a state in which the first target device is connected throughBLUETOOTH LE, the controller 180 may request enabling of an alternativecommunication element to be connected and a service from the firsttarget device.

Specifically, when a specific service provided by the first targetdevice is supported by BLUETOOTH BR/EDR (Basic Rate/Enhanced Data Rate),the controller 180 may perform handover from BLUETOOTH LE to BLUETOOTHBR/EDR.

More specifically, when a BLUETOOTH LE connection between the terminal100 and the first target device is established, the terminal 100 and thefirst target device may enter a connection state.

In addition, the terminal 100 may transmit a read request for requestingadditional information to the first target device when additionalinformation is necessary. The read request is a message for requestinginformation stored in the GATT database of the first target device.

In addition, the terminal may receive a read response from the firsttarget device in response to the read request. Therefore, the terminalmay receive the requested additional information.

Thereafter, the terminal 100 may transmit a write request message to thefirst target device in order to request ON of BLUETOOTH BR/EDR which isalternative communication technology to be connected and enabling of theservice.

The write request is a message for requesting writing of the handovercontrol point property of the GATT database of the first target device.At this time, the terminal 100 may request enabling of some or all ofthe services supportable by the first target device through the writerequest.

The first target device, which has received the write request message,may enter a BR/EDR page scan state. In addition, the terminal mayreceive a write response from the first target device in response to thewrite request.

The first target device turns on BLUETOOTH BR/EDR which is analternative communication element and enables the service according tothe request of the terminal 100. At this time, the first target devicemay enable some or all of the services requested by the terminal.

In addition, the terminal, which has received the write responsemessage, may enter the BR/EDR page state, transmit a page message to thefirst target device and establish a BLUETOOTH BR/EDR connection.

Thereafter, the terminal and the first target device may provideservices through BLUETOOTH BR/EDR.

Meanwhile, when a specific service provided by the first target deviceis supported by Wi-Fi Direct, the controller 180 may perform handoverfrom BLUETOOTH LE to Wi-Fi Direct.

More specifically, when a BLUETOOTH LE connection between the terminal100 and the first target device is established, the terminal 100 and thefirst target device may enter the connection state.

In addition, the terminal 100 may transmit a read request for requestingadditional information to the first target device when additionalinformation is necessary. The read request is a message for requestinginformation stored in the GATT database of the first target device.

In addition, the terminal may receive a read response from the firsttarget device in response to the read request. Therefore, the terminalmay receive the additional information requested by the terminal.

Thereafter, the terminal 100 may transmit a write request message to thefirst target device in order to request ON of Wi-Fi Direct which isalternative communication technology to be connected and enabling of theservice.

The write request is a message for requesting writing of the handovercontrol point property of the GATT database of the first target device.At this time, the terminal 100 may request enabling of some (e.g.,Miracast) or all of the services supportable by the first target devicethrough the write request.

The first target device, which has received the write request message,may enter a listen state of Wi-Fi Direct.

In addition, the terminal may receive a write response from the firsttarget device in response to the write request.

The first target device turns on Wi-Fi Direct which is an alternativecommunication element and enables the service according to the requestof the terminal 100. The first target device may enable some or all ofthe services requested by the terminal.

Meanwhile, the terminal, which has received the write response message,may enter the search state of Wi-Fi Direct, transmit a probe requestmessage to the first target device and establish a Wi-Fi Directconnection.

Thereafter, the first device 200 and the second device 300 may provideservices through Wi-Fi Direct.

FIG. 12 is a diagram showing a stack of the terminal 100 and the firsttarget device 200, and FIG. 13 is a diagram showing a detailed stack ofthe terminal 100.

In order to discover neighboring device supporting BLUETOOTH LE, theterminal may perform scanning and receive advertisement information froma target device.

Meanwhile, information on the network interface for providing a service(that is, necessary for connection and control) or a model ID may beacquired from the advertisement information.

In addition, the terminal may manage the model ID, recognize the targetdevice from the photographed image, and filter scanned target devices.Therefore, a target device to be connected may be determined.

Meanwhile, the GATT client of the terminal may request informationstored in the GATT database from the GATT server of the target device,and the GATT server may provide a service provided by the target deviceand the network interface for providing the service to the GATT Client.

The terminal may acquire the network interface for connection andcontrol of the determined target device.

When the network interface for connection and control of the targetdevice is not BLUETOOTH LE, handover may be requested.

In addition, the terminal may be connected to the target device usingthe network interface, handover of which has been requested, therebyproviding content such as audio to the target device or controlling thetarget device.

FIG. 14 is a diagram showing state change of the target device, and FIG.15 is a diagram showing state change of the terminal.

Referring to FIG. 14, when the target device is powered on, the targetdevice enters a BLE Ready state. In this state, BLE may transmit anadvertisement packet to the terminal.

In this case, the terminal (AI camera device) may scan a target devicein the BLE Ready state and acquire Model ID information included in theadvertisement packet through the scanned result.

When a service request is received from the terminal (AI camera device),the target device enters a Transport Ready state. In addition, thetarget device may turn on Transport capable of performing acorresponding service and enable the Transport to enter a connectionstandby state.

The target device enters a service state to perform a service requestedby a seeker. At this time, BLE continuously performs advertisingaccording to a request of the terminal (AI camera device). The terminal(AI camera device) may scan a target device in the service state.

When the service ends, the target device enters the BLE Ready state.

Referring to FIG. 15, when the terminal (AI camera device) is poweredon, the terminal may enter an initiating state.

When the camera operates, the terminal (AI camera device) transitions toa discovery state to perform scanning operation.

In some implementations, the terminal (AI camera device) may transitionfrom the initiating state to the discovery state by another operation(e.g., scan button selection, etc.) of the user to perform scanningoperation.

In order to use a provider and a specific service, the terminalexchanges information on Transport (BR/EDR, Wi-Fi, . . . ), throughwhich a connection is established based on a BLE GATT connection, andinformation on a used service, and enters a Transport ready state.

The terminal turns on Transport capable of performing a service desiredby the user in the Transport Ready state and enables the Transport toenter a connection standby state.

In addition, the terminal may perform the service in the Service stateand enter the Initiating state when the service ends.

FIG. 16 is a diagram showing a process of establishing connection withthe first target device 600 according to an embodiment of the presentdisclosure.

The process of scanning neighboring target devices by the terminal togenerate the list of scanned target devices may be performed before orafter the process of photographing the target device to estimate theidentification information of the target device.

For example, the terminal may first photograph the target device toestimate the identification information of the target device and thenscan the neighboring target devices to generate the list of scannedtarget devices.

In another example, the terminal may scan the neighboring target devicesto generate the list of scanned target devices and then photograph thetarget device to estimate the identification information of the targetdevice.

In order to reduce a time required for connection with the targetdevice, it may be advantageous that the list of neighboring targetdevices is already generated when the identification information of thetarget device is estimated. However, when scanning is continuouslyperformed for this purpose, power consumption may increase.

Accordingly, a method of minimizing a time required for connection witha target device while minimizing power consumption will be described.

In FIG. 16A, the controller may display an icon 1610 for a BLUETOOTH LEconnection.

In addition, when input of selecting the icon 1610 for the BLUETOOTH LEconnection is received, the controller may enable the photographingfunction of the camera and scan neighboring target devices.

When the photographing function of the camera is enabled, as shown inFIG. 16B, the controller may display a preview screen 1620 indicatingthe image received through the camera.

In this case, the user may capture the first target device 600 to becontrolled of the plurality of target devices. when the target device isincluded in the image received through the camera, the controller 180may perform photographing to acquire the photographed image of the firsttarget device. In this case, the controller 180 may automaticallyperform photographing even when user input of pressing the photographingbutton is not received.

Meanwhile, when a target device is included in the image receivedthrough the camera, the controller 180 may perform photographing at apredetermined time interval, thereby acquiring the photographed image ofthe first target device.

In this case, the controller 180 may perform photographing until arecognition rate becomes greater than a predetermined value based on therecognition rate determined by the machine learning model, to which thephotographed image is input.

Meanwhile, in a state in which the photographing function of the camerais enabled, that is, in a state in which the preview screen 1620 isdisplayed, the controller may scan neighboring target devices. Inaddition, the controller may receive advertisement information from theplurality of target devices and generate a list of the plurality oftarget devices using the received advertisement information.

After the list of the plurality of target devices is generated, when thelearned machine learning model estimates the identification informationof the first target device and outputs the estimated identificationinformation, the controller may acquire the first target devicecorresponding to the estimated identification information from the list.

That is, although enabling of the photographing function and scanningmay be performed simultaneously with selection of the icon 1610, a timedifference between enabling of the photographing function and actualphotographing occurs. Accordingly, the list of scanned devices is firstgenerated and then the estimated identification information is output.Accordingly, since the list of neighboring target devices is alreadygenerated when the identification information of the target device isestimated, it is possible to minimize a time required for connectionwith the target device.

In addition, the user may select the icon 1610 in order to connectBLUETOOTH LE. In addition, in the present disclosure, when the userselects the icon 1610, scanning is performed. Therefore, it isunnecessary to continuously perform scanning in order to generate thelist of neighboring target devices in advance.

Meanwhile, when the first target device corresponding to the estimatedidentification information is acquired from the list, as shown in FIG.16C, the controller may display a connection icon 1630 for connectionwith the first target device.

When input of pressing the connection icon 1630 is received, thecontroller may establish a connection between the terminal and the firsttarget device.

Meanwhile, the connection established by selection of the connectionicon 1630 is merely exemplary and the controller may automaticallyconnect the terminal with the first target device without user inputwhen the first target device corresponding to the estimatedidentification information is acquired from the list.

The controller may select a BLUETOOTH LE method or an alternativecommunication element according to the service provided by the firsttarget device, thereby connecting the terminal with the first targetdevice.

For example, when the first target device is a headset, the first targetdevice may provide an audio service. In this case, the controller maytransmit content stored in or reproduced on the terminal to the firsttarget device after establishing a BR/EDR connection with the firsttarget device.

In another example, when the first target device is a TV, the firsttarget device may provide a control service or a Miracast service by theterminal. In this case, the controller may transmit a control command orthe screen of the terminal to the first target device after establishinga Wi-Fi connection with the first target device.

In another example, when the first target device is a sensor, the firsttarget device may provide a sensing information provision serviceacquired by the first target device. In this case, the controller mayreceive sensing information from the first target device afterestablishing a BLUETOOTH LE connection with the first target device.

FIG. 17 is a diagram showing a control icon corresponding to thefunction of a target device.

When a first target device 500 is included in a preview screen 1710, thecontroller may enable a connection with the first target device.

When the connection with the first target device is enabled, thecontroller may display control icons 1721, 1722 and 1723 on the previewscreen 1710 corresponding to the functions of the first target device oran attribute icon capable of confirming the attributes of the firsttarget device.

In this case, the controller may display the control icons or theattribute icon on the preview screen 1720.

For example, when the first target device is a humidifier, thecontroller may display an icon for turning on/off the humidifier, anicon for controlling the strength of the humidifier, an icon forconfirming current humidity, and the like.

In this case, the terminal may transmit a control command to the firsttarget device, and the first target device may transmit the sensinginformation to the terminal.

In embodiments of the present disclosure, when the connection betweenthe terminal and the first target device is enabled, the control iconmay be displayed without executing a dedicated application (App) forcontrolling the first target device, such that the user may directlycontrol the first target device on the preview screen 1710.

That is, conventionally, a dedicated App of a device to be controlledhad to be executed, thus it forced users to install and use variousdifferent applications, where it may be difficult to adapt to differentuser interfaces between dedicated Apps. However, in embodiments of thepresent disclosure, the terminal may easily control a connected targetdevice without separately learning the control UX of each device.

FIG. 18 is a diagram illustrating a method of inputting a voice commandusing a terminal according to an embodiment of the present disclosure.

When a connection icon 1811 shown in FIG. 18A is selected, thecontroller may enable the camera function to display a preview screen1810 and enable a connection with a first target device 200 included inthe preview screen 1810.

When the connection with the first target device 200 is enabled, asshown in FIG. 18B, the controller may display text of the voice commandcorresponding to the function of the first target device.

In this case, the controller may display the text of the voice commandon the preview screen 1810.

In this case, voice commands frequently used by the user may bedisplayed on the screen. In addition, the text of the voice commandbased on information on currently reproduced content or the favoritechannel of the user may be displayed.

Meanwhile, as shown in FIG. 18C, the user may input the voice command tothe terminal 100 or may directly input the voice command to the firsttarget device 200.

When the voice command is input to the terminal 100, the terminal 100may transmit the voice command to the first target device 200.

In embodiments of the present disclosure, since available voice commandsmay be visually displayed and the user needs to learn the voicecommands, it is possible to improve user convenience.

The terminal described in this specification may include variousdevices, such as a mobile phone, a smartphone, a laptop, a digitalbroadcast terminal, a personal digital assistant (PDA), a portablemultimedia player (PMP), a navigation system, a slate PC, a tablet PC, alaptop or convertible PC, an ultrabook, a wearable device such as asmartwatch, smart glasses or a head mounted display (HMD), and the like.

However, it will be readily apparent to those skilled in the art thatthe configuration according to the embodiment described in thisspecification is applicable to a fixed terminal such as a digital TV, adesktop computer or a digital signage, in addition to the terminal.

In addition, the target device described in the present disclosure mayinclude a TV, a headset, a washing machine, a refrigerator, a speaker,an air purifier, an air conditioner, a humidifier, a denumidifier, anaudio player, a sensor, various other home appliances, smart speakers,smart displays, and the like.

A method of receiving shape information from target devices anddetermining a photographed target device using the shape informationwill be described with reference to FIGS. 19 to 35.

FIG. 19 is a diagram illustrating a wireless communication systemaccording to another embodiment of the present disclosure.

In the wireless communication system, a terminal 100, a plurality oftarget devices and a server 1900 may perform communication.

For the terminal 100 and the plurality of target devices, refer to thedescription of FIGS. 1 to 18C.

Meanwhile, the terminal 100 may communicate with the server 1900.

The server 1900 may include a first database 1910 and a second database1920.

A detailed information list including the detailed information of theplurality of target devices may be stored in the first database 1910.

Here, the detailed information may include at least one of the shapeinformation (shape ID) of the target device, model information (modelID), a supported alternative communication element (e.g., BLUETOOTHBR/EDR, Wi-Fi, Wi-Fi Direct, etc.), and type and distance estimationinformation of a providable service.

Meanwhile, a learned machine learning model may be stored in the seconddatabase 1920.

When the detailed information list is installed in the server 1900, theterminal may transmit at least one of the shape information and themodel information received from the target device to the server 1900. Inthis case, the server 1900 may search the detailed information listusing at least one of the received shape information and modelinformation. In addition, the server 1900 may acquire and transmitdetailed information corresponding to the received shape information andmodel information to the terminal.

In addition, when the trained machine learning model is installed in theserver 1900, the terminal may photograph the first target device andtransmit the photographed image to the server 1900. In this case, theserver 1900 may input the photographed image to the learned machinelearning model to acquire the estimated shape information of the firsttarget device. In addition, the server 1900 may transmit the estimatedshape information to the terminal.

The detailed information list and the trained machine learning model maybe stored in the terminal instead of the server.

Specifically, the storage unit of the terminal 100 may include a firstdatabase 1001 and a second database 1002.

The detailed information list including the detailed information of theplurality of target devices may be stored in the first database 1001.

The learned machine learning model may be stored in the second database1002.

When the detailed information list is installed in the terminal, thecontroller of the terminal may search the detailed information listusing the shape information and model information received from thetarget device. The controller of the terminal may acquire detailedinformation corresponding to the received shape information and modelinformation.

When the learned machine learning model is installed in the terminal,the controller of the terminal may photograph the first target device,input the photographed image to the learned machine learning model andacquire estimated shape information of the first target device.

FIG. 20 is a diagram showing a terminal communicating with a pluralityof target devices.

Referring to FIG. 20, the terminal may communicate with the plurality oftarget devices 2100, 2200, 2300 and 2400 in a communication range usinga BLUETOOTH Low Energy (BLE) method.

In addition, when the user captures an image of a first target device2000 to be controlled of the plurality of target devices, the controllermay perform photographing to acquire the photographed image of the firsttarget device 2000.

FIG. 21 is a diagram illustrating a method of operating a wirelesscommunication system according to an embodiment of the presentdisclosure.

The plurality of target devices 2000 and 2100 may transmit advertisementinformation to the terminal 100 (S2010 and S2020).

In addition, the terminal 100 may scan neighboring target devices(S2030).

This will be described with reference to FIG. 22.

For example, the first target device 2000 will be described. The firsttarget device 2000 in the advertising state may transmit advertisementinformation to neighboring devices for each advertisement event.

The terminal 100 may set a scanning parameter to perform scanningoperation in order to receive an advertisement packet from neighboringdevices in the standby state. In addition, the terminal 100 enters thescanning state in order to perform scanning operation, and receivesadvertisement packets from the neighboring devices during a scanningwindow duration. For example, the advertisement packets may be receivedfrom the plurality of target devices during the scanning duration of theterminal 100.

FIG. 22 is a diagram showing an example of the data format of anadvertisement packet.

Referring to FIG. 22, the advertisement packet has an AD structure, andthe terminal 100 may parse the advertisement packet in order to finddesired data.

Specifically, the plurality of target devices may include specific datain the advertisement packet in the AD structure through theabove-described advertisement parameter setting procedure.

For example, the plurality of target devices may configure theadvertisement packet in order to transmit transmission (Tx) power, TDS,data for providing distance estimation information, manufacturerinformation, which is information related to a manufacturer, through theadvertisement packet.

The advertisement packet may be composed of a plurality of ADstructures, and each AD structure may be composed of a length type value(LTV) structure.

A Length field may indicate the length of data, an Ad Type field mayindicate the type of included data, and AD Data may include actual data.

That is, each AD structure may include the Length field, the AD Typefield and the AD Data field.

Here, the AD type may indicate the type of the data included in AD Data.In addition, the AD Data field may include data according to AD Type,and may be composed of one or a plurality of LTV structures.

Meanwhile, the advertisement information transmitted by the targetdevice may include model information 2210 of the target device. Inaddition, the model information of the target device may be the model IDof the target device.

For example, the advertisement information transmitted by the secondtarget device 2100 may be the model ID of the second target device 2100which is the model information of the second target device 2100.

In another example, the advertisement information transmitted by thefirst target device 2000 may be the model ID of the first target device2000 which is the model information of the first target device 2000.

Here, the model ID may be a string obtained by combining aspects of theproduct, such as the function, color, appearance, product line, country,and the like, of the device, as a unique identifier created by themanufacturer to identify the device.

In this case, the plurality of target devices may include the modelinformation of each of the plurality of target devices in theadvertisement packet through the above-described advertisement parametersetting procedure.

In this case, the advertisement packet may include data indicating themodel ID of the target device. Specifically, an extended field includingdata indicating the model ID of the target device may be configuredusing Manufacturer AD type. In this case, the extended field may includean AI camera field such that the terminal 100 recognizes that the targetdevice is recognized through photographing, and may include a data field(Model ID for AI camera) indicating the model ID of the target device.

In addition, the advertisement information transmitted by the targetdevice may include shape information 2220 of the target device. Inaddition, the shape information of the target device may be a shape IDof the target device.

For example, the advertisement information transmitted by the secondtarget device 2100 may include the shape ID of the second target device2100, which is the shape information of the second target device 2100.

In another example, the advertisement information transmitted by thefirst target device 2000 may include the shape ID of the first targetdevice 2000, which is the shape information of the first target device2000.

Here, the shape ID may be a unique identifier applied to a shaperecognizable by the terminal, which has photographed the target device,when the target device is photographed.

In addition, the shape ID may indicate information for identifying thetarget device using the learned machine learning model, when the targetdevice is photographed and the photographed image is input to thelearned machine learning model.

In addition, the same shape ID may be given to even devices havingdifferent model IDs, when the machine learning model recognizes that thedevices having different model IDs have the same shape.

For example, when a specific speaker having a first model ID has acylindrical shape and a specific air purifier having a second model IDhas a cylindrical shape, the same shape ID may be given to the specificspeaker and the specific air purifier.

In this case, the plurality of target devices may include the modelinformation of each of the plurality of target devices in theadvertisement packet through the above-described advertisement parametersetting procedure.

In this case, the advertisement packet may include data indicating theshape ID of the target device. Specifically, an extended field includingdata indicating the shape ID of the target device may be configuredusing Manufacturer AD type. In this case, the extended field may includean AI camera field such that the terminal 100 recognizes that the targetdevice is recognized through photographing, and may include a data field(Shape ID for AI camera) indicating the shape ID of the target device.

In addition, the advertisement information may include the accessaddress of each of the plurality of target devices for accessing theplurality of target devices.

Meanwhile, the controller 180 may receive a plurality of pieces ofadvertisement information including the identification information ofthe plurality of target devices from the plurality of target devices.

Specifically, the terminal 100 is initially in the standby state. In thestandby state, the terminal 100 may not receive a message.

In addition, in the standby state, the terminal 100 may enter theadvertising sate, the scanning state or the initiating state.

In order to scan a neighboring target device supporting BLUETOOTH in thestandby state, the terminal 100 may perform scanning. In this case, theterminal 100 may enter the scanning state and receive the advertisementpacket from the neighboring target device.

In this case, the controller 180 may parse the received advertisementpacket and acquire the model ID, the shape ID and access address of thetarget device.

For example, the controller 180 may parse the advertisement packetreceived from the first target device 2000 and acquire the modelinformation of the first target device 2000 (the model ID of the firsttarget device 2000), the shape information of the first target device2000 (the shape ID of the first target device 2000) and the accessaddress.

Returning to FIG. 21, the controller 180 may control the camera tophotograph the first target device of the plurality of target devices(S2040). In addition, the controller 180 may acquire the shapeinformation, with which the first target device is labeled, using thephotographed image of the first target device (S2050).

This will be described with reference to FIG. 7 showing the trainingprocess of the neural network and FIG. 23 showing a method of estimatingshape information using a photographed image.

First, data for training of the neural network may be collected (S710).Here, the data is related to the plurality of target devices and mayinclude the photograph, shape ID, etc. of each of the plurality oftarget devices.

Meanwhile, the collected data may be preprocessed (S720). Specifically,the collected data may be processed and classified into a state suitablefor machine learning. In this case, an image format supported by alearning program, the number of data, data of each model, etc. may beclassified/collected. In addition, the data may be classified andcollected according to the shape ID. Here, the shape ID may correspondto the label of machine learning.

In addition, the collected data may be divided into training data andreview data.

Meanwhile, the trained machine learning model may be generated bytraining the neural network using the training data in a machinelearning manner (S730). Meanwhile, as the learning algorithm,Naive-Bayes, K-Nearest Neighbors, Logistic Regression, Support VectorMachine, etc. may be used in addition to the neural network.

Meanwhile, the training process of the neural network may be a processof labeling the image of the target device with shape information (shapeID) and performing training.

In order to improve a recognition rate, the images of the target devicephotographed at various angles and distances with various intensities ofillumination may be used as training data, and images obtained byphotographing a portion or the whole of the target device may be used astraining data.

Meanwhile, the training process of the neural network may be a processof labeling the image of each of the plurality of target devices withshape information (shape ID) and performing training.

The machine learning model generated by training may be a model trainedby labeling each of the plurality of target devices with shapeinformation. when an image including a target device is input, thetrained machine learning model may estimate and output the shapeinformation (shape ID), with which the target device is labeled.

When learning ends, the result learned using the review data, that is,the trained machine learning model, is confirmed (S740), and, whenaccuracy of the result of review exceeds a target value, the trainedmachine learning model may be installed in the terminal 100 (S750).

In this case, a program for implementing the machine learning model maybe stored in the storage unit.

Referring to FIG. 23A, the controller 180 may photograph the firsttarget device. In addition, when the photographed image of the firsttarget device is acquired, the controller 180 may estimate the shapeinformation of the first target device.

Specifically, the controller 180 may input the photographed image of thefirst target device to the trained machine learning model. In this case,the trained machine learning model may estimate the shape information ofthe first target device using the input image and output the estimatedresult. That is, the controller 180 may input the photographed image tothe trained machine learning model to acquire the estimated shapeinformation.

In addition, the controller may acquire the estimated distance from thefirst target device using the photographed image of the first targetdevice.

An example of calculating the estimated distance from the first targetdevice using the photographed image of the first target device will bedescribed with reference to FIG. 24.

FIG. 24 is a diagram illustrating an example of a method of acquiring adistance from a target device according to an embodiment of the presentdisclosure.

The controller 180 may receive input of the height H of the terminalfrom the user.

In addition, the controller 180 may recognize the floor in thephotographed image of the first target device and recognize a distancefrom the first target device located on the floor.

In addition, the controller 180 may calculate an angle (X degrees)between the terminal and the floor, on which the first target device islocated, using a direction sensor.

In addition, the controller 180 may calculate a distance from the firsttarget device using the acquired data.

Meanwhile, the description of FIG. 24 is merely exemplary and allmethods of calculating the distance from the target device using theimage, including time of flight information, may be used.

Meanwhile, the distance between the terminal 100 and the target devicemay be defined as distance 0.

Returning to FIG. 23, according to the above-described methods, thecontroller may estimate the shape ID of the first target device and thedistance between the first target device and the terminal using thephotographed image of the first target device.

When the shape information of the first target device and the distancefrom the first target device are estimated, as shown in FIG. 23b , thecontroller may generate estimated result information.

Returning to FIG. 21, the controller 180 may select a target device tobe connected with the terminal 100 from among the plurality of scannedtarget devices (S2060). In addition, the controller 180 may enable aconnection with the selected first target device (S2070).

Specifically, the controller 180 may select the first target device ofthe plurality of target devices as a target device to be connected withthe terminal 100 using the photographed image of the first targetdevice, the plurality of pieces of advertisement information receivedfrom the plurality of target devices and the distances from the targetdevices.

Assume that the terminal 100 receives the advertisement information ofthe first target device, the advertisement information of the secondtarget device, the advertisement information of the third target device,by scanning the first target device, the second target device and thethird target device.

In addition, assume that the first target device is photographed, thephotographed image of the first target device is input to the machinelearning model, and the machine learning model estimates the targetdevice in the image as the first target device and outputs the shapeinformation of the first target device.

The controller 180 may input the photographed image to the machinelearning model to estimate the shape information of the target deviceincluded in the photographed image.

Specifically, when the machine learning model is installed in theterminal, the controller 180 may input the photographed image to themachine learning model stored in the terminal to estimate the shapeinformation of the target device included in the photographed image. Inaddition, when the machine learning model is installed in the server,the controller 180 may transmit the photographed image to the server. Inthis case, the server may input the photographed image to the machinelearning model stored in the server to estimate the shape information ofthe target device included in the photographed image. In addition, thecontroller 180 may receive the estimated shape information from theserver.

Meanwhile, since the first target device is photographed, the machinelearning model may determine that the target device included in thephotographed image is the first target device, and output the shapeinformation of the first target device.

In addition, since the first target device is photographed, thecontroller may calculate the estimated distance from the first targetdevice using the photographed image.

When the plurality of target devices is scanned and the plurality ofpieces of advertisement information is received, the controller 180 maygenerate a list of the plurality of scanned target devices using theplurality of pieces of advertisement information.

For a target device which does not support the target device recognitionservice using photographing, shape information (shape ID) may not beincluded in advertisement information.

Accordingly, the controller 180 may generate a list of target devices,which has transmitted advertisement information including shapeinformation (shape ID). That is, the controller may generate a targetdevice list including the shape information (shape ID) of the firsttarget device and the shape information (shape ID) of the second targetdevice.

The controller 180 may acquire the distances from the target devices,which have transmitted the advertisement information.

Specifically, when the target device transmits advertisementinformation, the controller 180 may calculate the distance from thetarget device using the strength of the signal received from the targetdevice, for example, a received signal strength indication (RSSI).

For example, the controller may acquire a first distance from the firsttarget device using the strength of the signal received from the firsttarget device. In addition, the controller may acquire a second distancefrom the second target device using the strength of the signal receivedfrom the second target device.

Meanwhile, a log distance path loss model may be used through a methodof calculating a distance using a received signal. The presentdisclosure is not limited thereto and all methods of calculating thedistance using the received signal may be used.

Meanwhile, the transmission power Tx_Power of a wireless signal maydiffer between target devices.

Accordingly, the controller may acquire the transmission power Tx_Powerof the wireless signals of the plurality of target devices, andcalculate the distances from the plurality of target devices using thestrength of the signals received from the plurality of target devicesand the transmission power Tx_Power corresponding to the plurality oftarget devices.

Specifically, the plurality of target devices may transmit advertisementinformation including the transmission power Tx_Power of the wirelesssignals to the terminal.

For example, the first target device may transmit advertisementinformation including the shape information of the first target deviceand the transmission power Tx_Power of the first target device to theterminal.

In addition, the second target device may transmit advertisementinformation including the shape information of the second target deviceand the transmission power Tx_Power of the second target device to theterminal.

In this case, the controller 180 may acquire the first distance from thefirst target device using the strength of the signal received from thefirst target device and the transmission power of the first targetdevice.

In addition, the controller 180 may acquire the second distance from thesecond target device using the strength of the signal received from thesecond target device and the transmission power of the second targetdevice.

Meanwhile, the controller may enable a connection with the first targetdevice using the estimated shape information, the estimated distance,the shape information included in the plurality of pieces ofadvertisement information and the distances from the target devices.

Specifically, assume that the shape information estimated using thephotographed image is first shape information and a distance estimatedusing the estimated image is 1 m.

In addition, assume that first shape information is included in theadvertisement information transmitted by the first target device, andthe first distance from the first target device calculated using thestrength of the signal received from the first target device is 1.1 m.

In addition, assume that first shape information is included in theadvertisement information transmitted by the second target device, andthe second distance from the second target device calculated using thestrength of the signal received from the second target device is 2.0 m.

In addition, assume that second shape information is included in theadvertisement information transmitted by the third target device.

Since the estimated shape information is the first shape information andthe first shape information is received from the first target device andthe second target device, the controller may enable a connection withany one of the first target device and the second target device.

In addition, the controller may enable a connection with the firsttarget device based on proximity to the estimated distance.

Specifically, the first distance (1.1 m) is closer to the estimateddistance (1 m) than the second distance (2 m). Accordingly, thecontroller may enable a connection with the first target device.

In this case, the controller 180 may transmit the connect request to thefirst target device. In this case, the access address included in theadvertisement information transmitted by the first target device may beused.

Recently, technologies for identifying a subject using an AI camera haveappeared. However, these technologies may only identify the type of thesubject (a washing machine, a refrigerator, etc.), but may notaccurately identify the model of the target device.

However, in embodiments of the present disclosure, the identity of atarget device is estimated using machine learning and a target device tobe connected is determined using the shape information included in theadvertisement packet. When the user only captures the target device tobe connected using the camera, the photographed target device may beaccurately identified to establish a connection.

For example, when there are several electronic devices similarappearances, accuracy of identification through estimation of the targetdevice using machine learning may be significantly lowered. However, inthe present disclosure, the photographed target device may be accuratelyidentified using estimation of the distance using the image andcalculation of the distance using the strength of the signal along withthe shape ID, thereby establishing a connection.

FIG. 25 is a diagram showing a list of detailed information according toan embodiment of the present disclosure.

The controller may acquire the detailed information of the target deviceusing the advertisement information received from the target device.

Specifically, when the detailed information list 2510 including thedetailed information of the plurality of target devices is installed inthe terminal, the controller may search the detailed information liststored in the storage unit to acquire detailed information correspondingto the received advertisement information.

In addition, when the detailed information list including the detailedinformation of the plurality of target devices is installed in theserver, the controller 180 may transmit the received advertisementinformation to the server. In this case, the server may search thedetailed information list to acquire detailed information correspondingto the received advertisement information. In addition, the controller180 may receive the searched detailed information from the server.

Meanwhile, the detailed information may include a communication elementsupported by the target device, the type of a service provided by thetarget device or distance estimation information (transmission (Tx)power).

Accordingly, when the detailed information is searched for, the terminalmay establish a connection with the target device using thecommunication element supported by the target device and provide theservice supported by the target device.

Meanwhile, assume that the advertisement information does not includeshape information but includes only model information.

When only the model information is received, the terminal searches thelist including the detailed information of the plurality of targetdevices to find matched detailed information. Accordingly, search may beinefficient and slow.

FIG. 26 is a diagram illustrating a problem when advertisementinformation does not include model information but includes only shapeinformation.

When first shape information (0x01) is received from the first targetdevice (speaker) and second shape information (0x01) is received fromthe second target device (air purifier), the detailed information shownin FIG. 26 is searched for from the list including the detailedinformation of the plurality of target devices.

Accordingly, detailed information to be selected by the terminal may notbe specified.

In order to solve such a problem, as shown in FIG. 27, the advertisementinformation may include model information in addition to shapeinformation.

In FIG. 27, the list of the advertisement information received fromthree target devices is shown.

First advertisement information including first shape information 0x01,first model information P5570 and the address of the first target devicewas received from the first target device.

In addition, second advertisement information including first shapeinformation 0x01, second model information L220 and the address of thesecond target device was received from the second target device.

In addition, third advertisement information including first shapeinformation 0x02, third model information M7780 and the address of thethird target device was received from the third target device.

FIG. 28 is a diagram illustrating a method of operating a wirelesscommunication system according to an embodiment of the presentdisclosure.

FIG. 29 is a diagram showing a detailed information list.

FIGS. 30 and 31 are diagrams showing a list of searched detailedinformation.

S2810, S2820, S2830, S2840, 2850 and S2860 are equal to the abovedescription and thus a detailed description thereof will be omitted.

The peripheral device list shown in FIG. 27 is currently generated byscanning performed by the terminal.

Meanwhile, the controller may search the detailed information list usingthe received shape information and model information (S2870) and acquiredetailed information corresponding to the received shape information andmodel information.

First, the controller may acquire detailed information corresponding tothe shape information received using the received shape information.

Specifically, since the first target device transmits the first shapeinformation 0x01, the second target device transmits the first shapeinformation 0x01, and the third target device transmits the second shapeinformation 0x02, the controller may preferentially search for detailedinformation 2910, 2920, 2930 and 2940 including the first shapeinformation and the second shape information from the detailedinformation list 2510.

Then, the controller may search for the first detailed information 2910corresponding to the first model information P5570 and the seconddetailed information 2920 corresponding to the second model informationL220 from the detailed information 2910 and 2920 corresponding to thefirst shape information 0x01 using the received model information. Inaddition, the controller may acquire the third detailed information 2930corresponding to the third model information M7780 from the detailedinformation 2930 and 2940 corresponding to the second shape information0x02 using the received model information.

Meanwhile, since the first target device has been photographed, themachine learning model has estimated the first shape information 0x01.

Accordingly, the controller may acquire the detailed information 2910and 2920 corresponding to the estimated first shape information 0x01 ofthe acquired detailed information 2910, 2920 and 2930 (S2880). Thedetailed information 2910 and 2920 corresponding to the first shapeinformation 0x01 is shown in FIG. 30.

Meanwhile, the distance estimated using the photographed image is 1 m,the first distance from the first target device calculated using thestrength of the signal received from the first target device is 1.1 m,and the second distance from the second target device calculated usingthe strength of the signal received from the second target device is 2.0m.

Meanwhile, although the transmission power of the first target deviceand the transmission power of the second target device are respectivelyreceived from the target devices, the present disclosure is not limitedthereto and the transmission power may be extracted from the detailedinformation list 2510. For example, when the first identificationinformation and the first model information are received from the firsttarget device, the controller may acquire detailed informationcorresponding to the first identification information and the firstmodel information. In addition, the detailed information correspondingto the first identification information and the first model informationmay include the transmission power of the first target device.

That is, since the estimated distance is close to the first distance,the controller may select the first target device as a target device tobe connected.

In this case, the controller may select the detailed information 2910corresponding to the first model information P5570 received from thefirst target device. The finally searched detailed information 2910 isshown in FIG. 31.

Meanwhile, the controller may acquire the communication element andservice provided by the first target device through the searcheddetailed information 2910, connect the terminal with the first targetdevice using the communication element provided by the first targetdevice, and then provide the service.

For example, when the first target device is a speaker, thecommunication element provided by the first target device may beBLUETOOTH and the service provided by the first target device may be aspeaker service.

In this case, the controller may transmit content, which is stored inthe terminal or is being reproduced, to the first target device throughBLUETOOTH (BT).

FIG. 32 is a diagram showing information included in a detailedinformation list and information acquired by a controller according toan embodiment of the present disclosure. FIG. 29 will also be referredto.

The detailed information list may include detailed information 3210including shape information, model information, a connection method, aservice type and transmission power.

Meanwhile, when the shape information and the model information arereceived from the target device, the controller may search for thedetailed information 3220 corresponding to the received shapeinformation and model information. In addition, the searched detailedinformation may further include address information included in theadvertisement information and distance information calculated using thestrength of the received signal. The detailed information correspondingto the received advertisement information may be acquired in thismanner.

Meanwhile, the controller may select the detailed information 3230corresponding to the estimated shape information.

In addition, the controller may acquire the detailed information 3240 ofa target device closer to the terminal. In this case, the detailedinformation 3240 may further include distance information distance_1between the terminal and the target device, the estimated distancedistance_0 from the photographed target device, and proximityproximity_0 between the terminal and the target device.

According to embodiments of the present disclosure, there are many stepsof connecting the target device with the terminal and providing aservice. However, when the target device is captured by the camera, aconnection is immediately established to provide a service (soundtransmission of the terminal and sound output of the target device),thereby providing a best service to the user.

In the present disclosure, information on the connection method and theservice is acquired in advance using the shape information and the modelinformation, thereby significantly shortening a time required forconnection and service provision.

For example, when the user selects an icon 1610 for BLUETOOTH LEconnection of FIG. 16A, the terminal performs scanning. This process maybe finished before the machine learning model estimates the shapeinformation after the user performs photographing. In addition, sincethe detailed information corresponding to the model information and theshape information is searched for in advance, the number of objects tobe compared with the estimated shape information is significantlyreduced. Therefore, the present disclosure may significantly reduce atime required to provide the service after the user captures the targetdevice.

FIG. 33 is a view illustrating a method of establishing a connectionwith a target device according to the embodiment of the presentdisclosure.

When input of establishing a connection with a target device is receivedwhile a specific application is executed and a service provided by thefirst target device supports a specific application, the controller mayestablish a connection with the first target device and provide aservice without separate input.

For example, as shown in FIG. 33A, assume that an icon for a BLUETOOTHLE connection is selected while a music application is executed.

As shown in FIG. 33B, when the first target device determined as thetarget device to be connected is a speaker 2000, the speaker 2000 maysupport the music application to output music.

In this case, the controller may establish a connection with the speaker2000 without separate input and immediately transmit content forproviding a sound output service. In this case, the speaker 2000 mayalso output sound without separate input. In addition, the controllermay display a handover icon 3510 indicating that handover has beenperformed.

In contrast, as shown in FIG. 33C, when the first target devicedetermined as the target device to be connected is an air purifier 2100,the air purifier 2100 may not output music through the musicapplication.

In this case, the controller may display a connection icon 3520. Wheninput of selecting the connection icon 3520 is received, the controllermay control a communication unit to establish a connection with the airpurifier.

In FIG. 34 to FIG. 44, this section describes how to select a targetdevice to connect to the mobile device when the shape information anddistance information are the same between first and second devices.

Specifically, in the above embodiment, when the shape information of thefirst target device and the second target device are the same, thecontroller may calculate a first distance from the first target deviceand a second distance from the second target device. When the firstdistance is close to the estimated distance, the controller activatesthe connection with the first target device.

In this case, two problems may occur.

Specifically, when the user is currently illuminating the first targetdevice, or when there is a second target device with the same shape anddistance as the first target device, the mobile terminal does not knowwhich target device among the first target device and the second targetdevice to connect therewith.

Further, when the user is currently illuminating the first target deviceand the second target device, and when first and second target deviceshave the same shape information and when a distance between the firsttarget device and the mobile terminal is substantially equal to adistance between the second target device and the mobile terminal, themobile terminal does not know which target device among the first targetdevice and the second target device to connect therewith.

Therefore, the solution to this problem is described with reference toFIG. 34 to FIG. 44.

In the following, an concept of an action ID is presented.

In this connection, the action ID may be information including a schemeenabling the mobile terminal to identify a target device when the mobileterminal cannot distinguish the device only using the shape informationand the model information.

In more detail, the action ID may include a scheme for identifying atarget device. For example, the first action ID among the action IDs mayinclude a scheme for identifying a target device using the color of theoptical indicator (LED), the second action ID among the action IDs mayinclude a scheme for identifying a target device using the light outputpattern from the optical indicator LED. The third action ID among theaction IDs may include a scheme for identifying a target device using animage pattern output by the optical indicator LED. The fourth action IDmay include a scheme for identifying a target device using a buttonpressing operation. The fifth action ID may include a scheme foridentifying a target device using a control operation of some componentsof the target device.

Further, the action ID may include action information. In thisconnection, the action information may include means for specifying thetarget device by distinguishing the target devices. For example, thefirst action information of the action information may include thespecific color (e.g. red) displayed by the optical indicator (LED), andthe second action information may include the specific pattern (e.g.,blinking once every second) outputted by the optical indicator LED. Thethird action information may include a specific image pattern (e.g., aspecific QR code) output by the optical indicator (LED), and the fourthaction information may include a button pressing operation on the targetdevice. The fifth action information may include a control operation(for example, opening of a cover by a user) of some components of thetarget device.

In one example, the action information may be included in the action IDand received from the target device, or may be received from the targetdevice separately from the action ID.

Further, the action information may be obtained from an externaldatabase. In this case, the controller of the mobile terminal sends themodel ID to the external database. Action information corresponding tothe transmitted model ID may be received by the controller.

Further, the action information may be loaded into the memory of themobile terminal when the product is released.

FIG. 34 illustrates an operating method of a wireless communicationsystem according to an embodiment of the present disclosure.

The plurality of target devices 2000 and 2100 may transmit advertisementinformation to the mobile terminal 100 S3410 and S3420.

Further, the mobile terminal 100 may scan the surrounding target devicesS3430.

In one example, the advertisement information transmitted from thetarget device may further include an action ID of the target device.

For example, the advertisement information transmitted from the firsttarget device 2000 may include an action ID of the first target device2000. In another example, advertisement information transmitted from thesecond target device 2100 may include an action ID of the second targetdevice 2100.

In one example, the controller of the mobile terminal 100 may obtainshape information and estimated distances of the target devices S3440.

Specifically, when the predicted shape information is first shapeinformation and the first shape information is received from the firsttarget device and the second target device, the controller may obtain afirst distance of the first device using a strength of a signal receivedfrom the first target device, and a second distance of the second deviceusing a strength of a signal received from the second target device.

In one example, when the shape information of the first target deviceand the second target device of the plurality of target devices are thesame and when the first distance between the first target device and themobile terminal is substantially equal to the second distance betweenthe second target device and the mobile terminal, the controller maysend the operation execution command thereto S3450.

In this connection, the first distance and the second distance aresubstantially equal to each other may not mean that the first distanceand the second distance are exactly equal to each other but may meanthat the first distance and the second distance are substantially equalto each other such that whether the first distance or second distance iscloser to an estimated distance is not determined.

Further, based on the operation of the target device corresponding tothe operation execution command, the controller of the mobile terminalmay activate the connection with the first target device S3460.Specifically, based on an operation of at least one of the first targetdevice or the second target device, the controller of the mobileterminal may activate the connection with the first target device.

This will be described with reference to FIG. 35 to FIG. 37.

FIG. 35 illustrates a method of activating a connection with a targetdevice when the user is illuminating a target device according to anembodiment of the present disclosure.

The controller of the mobile terminal 100 may transmit an operationexecution command to the first target device 2000 and the second targetdevice 2100.

In this case, the first target device and the second target device mayperform an operation corresponding to their action information.

For example, it is assumed that the first action informationcorresponding to the first target device is that the optical indicatorof the first device outputs red light. The first target device, uponreceiving the operation execution command, may control the opticalindicator thereof to output red light.

In another example, it is assumed that the second action informationcorresponding to the second target device is that the optical indicatorthereof outputs green light. The second target device, upon receivingthe operation execution command, may control the optical indicatorthereof to output green light.

In one example, as shown in FIG. 35, when the target device 3510included in the captured image performs an operation corresponding tothe first action information of the first target device 2000, thecontroller of the mobile terminal 100 may activate a connection with thefirst target device.

In more detail, the controller of the mobile terminal 100 may comparethe operation of the target device 3510 included in the captured imagewith the first action information corresponding to the first targetdevice 2000 and the second action information corresponding to thesecond target device 2100.

When the operation of the target device 3510 included in the capturedimage corresponds to the first action information corresponding to thefirst target device 2000, the controller of the mobile terminal 100 mayactivate the connection with the first target device 2000.

For example, the first action information corresponding to the firsttarget device 2000 indicates red light emission from the LED, while thesecond action information corresponding to the second target device 2100indicates green light emission from the LED. Thus, when the operation ofthe target device 3510 included in the captured image indicate red lightemission through the LED, the controller of the mobile terminal mayactivate the connection with the first target device.

The user illuminating the first target device 2000 using the mobileterminal means that the user's intention is to connect with the firsttarget device 2000. According to the present disclosure, despite thefirst and second target devices among the plurality of target deviceshave the same shape information and the distance between the firsttarget device and the mobile terminal is substantially equal to thedistance between the second target device and the mobile terminal, theuser may activate a connection between the mobile terminal and one ofthe first target device and second device selected based on the user'sintention.

FIG. 36 illustrates a method for activating a connection with a targetdevice when the user is illuminating two target devices according to anembodiment of the present disclosure.

The controller of the mobile terminal 100 may transmit an operationexecution command to the first target device 2000 and the second targetdevice 2100.

In this case, the first target device and the second target device mayperform an operation corresponding to their action information.

For example, it is assumed that the first action informationcorresponding to the first target device is that the optical indicatoroutputs red light. The first target device, upon receiving the operationexecution command, may control the optical indicator to output redlight.

In another example, it is assumed that the second action informationcorresponding to the second target device is that the optical indicatoroutputs green light. The second target device, upon receiving theoperation execution command, may control the optical indicator to outputgreen light.

In one example, when, as shown in FIG. 36, two target devices 3610 and3620 included in the captured image perform the operation correspondingto the first action information of the first target device 2000 and theoperation corresponding to the second action information of the secondtarget device 2100, respectively, the controller of the mobile terminal100 may activate the connections with the first target device and thesecond target device.

In detail, the controller of the mobile terminal 100 may compare theoperations of the two target devices 3610 and 3620 included in thecaptured image with the first action information corresponding to thefirst target device 2000 and the second action information correspondingto the second target device 2100.

When the operation of one 3610 of the two target devices 3610 and 3620included in the captured image corresponds to the first actioninformation corresponding to the first target device 2000, and when theoperation of the other target device 3620 among the two target devices3610 and 3620 included in the captured image corresponds to the secondaction information corresponding to the second target device 2100, thecontroller of the mobile terminal 100 may activate the connections withthe first target device 2000 and the second target device 2100.

For example, the first action information corresponding to the firsttarget device 2000 indicates red light emission through the LED. Thesecond action information corresponding to the second target device 2100indicates green light emission through the LED. When the operation ofone 3610 of the two target devices 3610 and 3620 included in thecaptured image is red light emission through the LED and when theoperation of the other 3620 of the two target devices 3610 and 3620included in the captured image is green light emission through the LED,the controller of the mobile terminal may activate the connections withthe first target device and the second target device.

The fact that the user illuminates the first target device 2000 and thesecond target device 2100 using the mobile terminal may mean that theuser's intention is to connect with the first target device 2000 and thesecond target device 2100. According to the present disclosure, there isan advantage that the mobile terminal may connect with the target deviceaccording to the user's intention.

In one example, when the first target device 2000 and second targetdevice 2100 are connected to the mobile terminal, the controller of themobile terminal may control the first target device 2000 and the secondtarget device 2100 in a grouped manner. For example, when a volume upinput is received from the user, the controller of the mobile terminalmay transmit a volume up command to two audio output devices (that is,the first target device 2000 and the second target device 2100).

In one example, FIG. 36 describes the activation of the connection withboth target devices, but the present disclosure is not limited thereto.The controller may activate the connection with the target device asselected by the user among the two target devices. This is explainedwith reference to FIG. 37.

FIG. 37 illustrates a method of activating a connection with one targetdevice when the user is illuminating two target devices, according to anembodiment of the present disclosure.

The controller of the mobile terminal 100 may transmit an operationexecution command to the first target device 2000 and the second targetdevice 2100.

In this case, the first target device and the second target device mayperform an operation corresponding to their action information.

For example, it is assumed that the first action informationcorresponding to the first target device is that the optical indicatoroutputs red light. The first target device, upon receiving the operationexecution command, may control the optical indicator to output redlight.

In another example, it is assumed that the second action informationcorresponding to the second target device is that the optical indicatoroutputs green light. The second target device, upon receiving theoperation execution command, may control the optical indicator to outputgreen light.

In one example, when two target devices 3610 and 3620 included in thecaptured image respectively perform an operation corresponding to thefirst action information of the first target device 2000 and anoperation corresponding to the second action information of the secondtarget device 2100, the controller of the mobile terminal 100 mayactivate the connection with the target device 2100 as selected by theuser.

In more detail, the controller of the mobile terminal 100 may receive aninput for selecting one of the first target device and the second targetdevice from the user. Further, when the operation of the selected targetdevice is an operation corresponding to the second action information,the controller may activate the connection with the second targetdevice.

More specifically, it is assumed that a second object 3620 is selectedfrom two objects 3610 and 3620 displayed on the screen. In this case,the controller of the mobile terminal 100 may determine that theoperation of the second object 3620 is to output green light.

In this case, the operation of the second object 3620 is the same as thesecond action information of the second target device. Therefore, thecontroller of the mobile terminal 100 may determine that the secondtarget device is selected by the user and then may activate theconnection with the second target device.

Although the user illuminates the first target device 2000 and thesecond target device 2100 using the mobile terminal, and the firsttarget device 2000 and the second target device 2100 are locatedadjacent to each other, the controller may activate the connection withthe second target device because the user may want to connect with thesecond target device 2100. According to the present disclosure, there isan advantage that the controller may connect with the target deviceaccording to the user's intention.

In one example, the controller of the mobile terminal may detect theoperations of the target devices based on the received action ID.

For example, when the action ID specifies a scheme of using the color ofthe optical indicator as a scheme for identifying the target device, thecontroller of the mobile terminal detects the color output by theoptical indicator of the target device. The detected color may becompared with the action information of the target device.

In another example, the action ID specifies a scheme of using the lightoutput pattern of the optical indicator as a scheme for identifying thetarget device, the controller of the mobile terminal detects the lightoutput pattern output by the optical indicator of the target device. Thedetected pattern may be compared with the action information of thetarget device.

FIG. 38 illustrates an operating method of a wireless communicationsystem according to another embodiment of the present disclosure.

The plurality of target devices 2000 and 2100 may transmit advertisementinformation to the mobile terminal 100 S3710 and S3720.

Further, the mobile terminal 100 may scan the surrounding target devicesS3730.

In one example, the controller of the mobile terminal 100 may obtainshape information and estimated distances of target devices S3740.

Further, when first and second target devices among the plurality oftarget devices have the same shape information and when a distancebetween the first target device and the mobile terminal is substantiallyequal to a distance between the second target device and the mobileterminal, the operation execution command may be sent thereto S3750.

In this case, the controller of the mobile terminal may activate theconnection with the first target device based on an operation of atleast one of the first target device or the second target device.

In other words, the controller of the mobile terminal receives theaction information from the target device and then activates theconnection with the target device according to the operation and actioninformation of the captured target device S3750.

This will be described with reference to FIG. 39 to FIG. 41.

FIG. 39 illustrates a method of activating a connection with a targetdevice when the user illuminates one target device according to anembodiment of the present disclosure.

The controller of the mobile terminal 100 may transmit an operationexecution command to the first target device 2000 and the second targetdevice 2100.

In this case, each of the first target device and the second targetdevice may perform an operation corresponding to its action information.Further, each of the first target device and the second target devicemay transmit its action information to the mobile terminal.

In one example, the action information of the first target device andthe second target device may be unique information allocated to each ofthe first target device and the second target device.

However, the present disclosure is not limited thereto. The actioninformation of the first target device and the second target device maybe designated by the mobile terminal. Specifically, the controller ofthe mobile terminal 100 transmits the first action informationcorresponding to the first target device to the first target devicetogether with the operation execution command and transmits, along withthe operation execution command, the second action informationcorresponding to the second target device to the second target device.

For example, the controller of mobile terminal 100 transmits the firstaction information indicating that the first target device is to beturned on thereto and transmits the second action information indicatingthat the second device is not to be turned to the second target device.

In one example, each OF the first target device and second target deviceperforms the operation. Action information corresponding to theperformed operation may be transmitted to the mobile terminal. In thiscase, the controller of the mobile terminal 100 receives the firstaction information performed on the first target device from the firsttarget device, and the second action information performed on the secondtarget device from the second target device.

In one example, as shown in FIG. 39, when the target device 3910included in the captured image performs an operation corresponding tothe first action information of the first target device 2000, thecontroller of the mobile terminal 100 determine that the target deviceincluded in the captured image is the first target device. Then, thecontroller may activate the connection with the first target device.

In more detail, the controller of the mobile terminal 100 may comparethe operation of the target device 3910 included in the captured imagewith the first action information corresponding to the first targetdevice 2000 and the second action information corresponding to thesecond target device 2100.

When the operation of the target device 3910 included in the capturedimage corresponds to the first action information corresponding to thefirst target device 2000, the controller of the mobile terminal 100 mayactivate the connection with the first target device 2000.

For example, the first action information corresponding to the firsttarget device 2000 indicates lighting of the LED. The second actioninformation corresponding to the second target device 2100 indicatesnon-lighting of the LED. Thus, the controller of the mobile terminal mayactivate the connection with the first target device.

The user illuminating the first target device 2000 using the mobileterminal means that the user's intention is to connect with the firsttarget device 2000. Thus, according to the present disclosure, despitethe first and second target devices among the plurality of targetdevices have the same shape information and when a distance between thefirst target device and the mobile terminal is substantially equal to adistance between the second target device and the mobile terminal, thecontroller may activate the connecting with the target device betweenthe first and second devices as selected based on the user's intention.

In one example, in FIG. 39, it is described that the action informationindicates whether the optical indicator or the LED is turned on or off.FIG. 40 illustrates the case where action information is associated withthe light output color. Further, FIG. 40 illustrates a method foractivating a connection with a target device when the user isilluminating two target devices.

FIG. 40 is a diagram illustrating a method for activating a connectionwith a target device when the user is illuminating two target devices.

The controller of the mobile terminal 100 may transmit an operationexecution command to the first target device 2000 and the second targetdevice 2100.

In this case, each of the first target device and the second targetdevice may perform an operation corresponding to its action information.Further, each of the first target device and the second target devicemay transmit its action information to the mobile terminal.

In one example, each of the action information of the first targetdevice and the second target device may be unique information allocatedto each of the first target device and the second target device. Forexample, the first action information unique to the first target devicemay indicate outputting red light, and the second action informationunique to the second target device may indicate outputting green light.

However, the present disclosure is not limited thereto. The actioninformation of the first target device and the second target device maybe designated by the mobile terminal. For example, the controller of themobile terminal 100 transmits first action information to the firsttarget device indicating that the device outputs red light. Thecontroller of the mobile terminal 100 transmits second actioninformation to the first target device indicating that the deviceoutputs green light.

In one example, when action information is specified by the mobileterminal, the controller of the mobile terminal may specify actioninformation based on the action ID. For example, when each of the actionID of the first target device and the action ID of the second targetdevice specifies a scheme of using the color of the optical indicator inorder to identify each device, the controller of the mobile terminal maytransmit first action information to output light of a first color tothe first target device and second action information to output light ofa second color to the second target device.

In one example, each of the first target device and second target deviceperform an operation. Action information corresponding to the performedoperation may be transmitted to the mobile terminal. In this case, thecontroller of the mobile terminal 100 receives the first actioninformation performed on the first target device from the first targetdevice, and the second action information performed on the second targetdevice from the second target device.

In one example, when as shown in FIG. 40, one 4010 of the two targetdevices 4010 and 4020 included in the captured image performs anoperation corresponding to the first action information, while the othertarget device 4020 thereof performs an operation corresponding to thesecond action information, the controller of the mobile terminal 100 maydetermine that one target device 4010 is the first target device 2000and the other target device 4020 is the second target device.

In detail, the controller of the mobile terminal 100 may compare theoperations of the two target devices 4010 and 4020 included in thecaptured image with the first action information corresponding to thefirst target device 2000 and the second action information correspondingto the second target device 2100 respectively.

When the operation of one 4010 of the two target devices 4010 and 4020included in the captured image corresponds to the first actioninformation of the first target device 2000, the controller of themobile terminal may determine that one target device 4010 is the firsttarget device 2000.

Further, when the operation of the other target device 4020 of the twotarget devices 4010, 4020 included in the captured image corresponds tothe second action information of the second target device 2100, thecontroller of the mobile terminal 100 may determine that the othertarget device 4020 is the second target device 2100.

Further, when one target device 4010 is selected by the user, thecontroller of the mobile terminal may activate the connection with thefirst target device 2000.

Thus, according to the present disclosure, although the shapes of thetarget devices are the same, and distances between the mobile terminaland the target devices are same, the mobile terminal may connect with atarget device meeting the user's intention.

In one example, in FIG. 40, the action information indicates the lightoutput color of the optical indicator. However, the present disclosureis not limited thereto. The action information may indicate a lightoutput pattern.

For example, the first action information corresponding to the firsttarget device may indicate single time blinking per second, and thesecond action information corresponding to the second target device maybe two times blinking per second.

Further, the action information may be an image pattern. This will bedescribed with reference to FIG. 41.

FIG. 41 is a diagram for explaining a method of identifying a targetdevice using an image pattern.

The controller of the mobile terminal 100 may transmit an operationexecution command to the first target device 2000 and the second targetdevice 2100.

In this case, each of the first target device and the second targetdevice may perform an operation corresponding to its action information.Further, each of the first target device and the second target devicemay transmit its action information to the mobile terminal.

In one example, the action information of the first target device andthe second target device may be unique information allocated to thefirst target device and the second target device, respectively. However,the present disclosure is not limited thereto, and action information ofthe first target device and the second target device may be designatedby the mobile terminal.

In one example, each of the first target device and second target deviceperforms an operation. Action information corresponding to the performedoperation may be transmitted to the mobile terminal.

For example, the first target device may output a first QR code throughthe optical indicator of the first target device. In this case, thefirst target device may send the first QR code to the mobile terminal.

Further, the second target device may output a second QR code throughthe optical indicator of the second target device. In this case, thesecond target device may transmit the second QR code to the mobileterminal.

In one example, as shown in FIG. 41, when one 4010 of the two targetdevices 4010 and 4020 included in the captured image performs anoperation corresponding to the first action information and the othertarget device 4020 thereof performs an operation corresponding to thesecond action information, the controller of the mobile terminal 100 maybe determined that the one target device 4020 is the first target device2000 and the other target device 4020 is the second target device 2100.

In more detail, the controller of the mobile terminal 100 may compareimage patterns output from the two target devices 4010 and 4020 includedin the captured image with the first action information corresponding tothe first target device 2000 and the second action informationcorresponding to the second target device 2100.

When the image pattern from one 4010 of the two target devices 4010 and4020 included in the captured image corresponds to the first QR code ofthe first target device 2000, the controller of the mobile terminal maydetermine that the one target device 4010 is the first target device2000.

Further, when the image pattern from the other target device 4020 of thetwo target devices 4010 and 4020 included in the captured imagecorresponds to the second QR code of the second target device 2100, thecontroller of the mobile terminal 100 may determine that the othertarget device 4020 is the second target device 2100.

FIG. 42 illustrates an operating method of a wireless communicationsystem according to another embodiment of the present disclosure.

The plurality of target devices 2000 and 2100 may transmit advertisementinformation to the mobile terminal 100 S4210 and S4220.

Further, the mobile terminal 100 may scan the surrounding target devicesS4230.

In one example, the controller of the mobile terminal 100 may obtainshape information and estimated distances of target devices S4240.

Further, when the shape information of the first target device and thesecond target device are substantially the same information and thefirst and second distances are substantially the same, the controller ofthe mobile terminal may display an action request S4250.

Further, the controller of the mobile terminal receives the actioninformation and may activate the connection with the target deviceaccording to the action request and the action information.

This will be described with reference to FIG. 43 to FIG. 44.

FIG. 43 illustrates a method of activating a connection with a targetdevice when the user illuminates one target device according to anembodiment of the present disclosure.

In the case where the shape information of the first target device andthe second target device among the plurality of target devices are thesame and the distance between the first target device and the mobileterminal and the distance between the second target device and themobile terminal are the same, the controller of the mobile terminal 100may control a display to display the action request.

In this case, the action request may be selected based on the action ID.For example, when the action ID of the first target device or the actionID of the second target device specifies a scheme of pressing a buttonas a scheme of identifying a device, the controller of the mobileterminal may display the action request requesting a button press.

In one example, the user may press a button on the first target devicein response to an action request.

In this case, the controller of the first target device 2000 maytransmit action information corresponding to the action request to themobile terminal 100. For example, the controller of the first targetdevice 2000 detects that the button has been pressed and then transmitsaction information indicating that the button is pressed to the mobileterminal 100.

In one example, when action information corresponding to an actionrequest is received from the first target device, the controller of themobile terminal 100 may activate the connection with the first targetdevice.

For example, when the mobile terminal 100 displays the action requestrequesting a button press and when action information indicating thatthe button is pressed is received from the first target device, thecontroller of the mobile terminal 100 may activate the connection withthe first target device.

In one example, the action information may indicate a control operationof some components of the target device. For example, the actioninformation may indicate an opening of a cover of the target device.

Specifically, the controller of the mobile terminal may display anaction request requesting to open the cover of the target device.

In one example, the user may open the cover of the first target devicein response to the action request.

In this case, the controller of the first target device 2000 maytransmit action information corresponding to the action request to themobile terminal 100. For example, the controller of the first targetdevice 2000 detects that the cover is open, and sends action informationindicating that the cover is open to the mobile terminal 100.

Then, when the action information corresponding to the action request isreceived from the first target device, the controller of the mobileterminal 100 may activate the connection with the first target device.

FIG. 44 illustrates a method for activating a connection with one of twotarget devices according to an embodiment of the present disclosure.

In the case where the shape information of the first target device andthe second target device among the plurality of target devices are thesame and the distance between the first target device and the mobileterminal and the distance between the second target device and themobile terminal are the same, the controller of the mobile terminal 100may control the display to display the action request.

In one example, the user wants to activate the first target device.Therefore, the user may press the button of the first target device inresponse to the action request.

In this case, the controller of the first target device 2000 maytransmit action information corresponding to the action request to themobile terminal 100. Then, when the action information corresponding tothe action request is received from the first target device, thecontroller of the mobile terminal 100 may activate a connection with thefirst target device among the first target device 2000 and the secondtarget device 2100.

The present disclosure may also be embodied as non-transitory computerreadable codes on a computer readable recording medium. The computerreadable recording medium is any data storage device that may store datawhich may be thereafter read by a computer system. Examples of thecomputer readable recording medium include HDD (Hard Disk Drive), SSD(Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, amagnetic tape, a floppy disk, an optical data storage device, the othertypes of storage mediums presented herein, and combinations thereof. Thecomputer may include the controller 180 of the terminal. The aboveexemplary embodiments are therefore to be construed in all aspects asillustrative and not restrictive. The scope of the disclosure should bedetermined by the appended claims and their legal equivalents, not bythe above description, and all changes coming within the meaning andequivalency range of the appended claims are intended to be embracedtherein.

What is claimed is:
 1. A mobile terminal comprising: a communicator forcommunicating with a plurality of target devices in a Bluetooth LowEnergy (BLE) manner; a camera for capturing an image; and a controllerconfigured for: receiving a plurality of advertisement informationrespectively including a plurality of shape information of the pluralityof target devices from the plurality of target devices; and when firstand second target devices among the plurality of target devices have thesame shape information and when a distance of the first target device isequal to a distance of the second target device, activating a connectionbetween the mobile terminal and the first target device based on anoperation of at least one of the first target device or the secondtarget device.
 2. The mobile terminal of claim 1, wherein the controlleris further configured for: when first and second target devices amongthe plurality of target devices have the same shape information and whena distance of the first target device is equal to a distance of thesecond target device, transmitting an operation execution command; andactivating a connection between the mobile terminal and the first targetdevice based on an operation of a target device corresponding to theoperation execution command.
 3. The mobile terminal of claim 2, whereinthe controller is further configured for: transmitting the operationexecution command; and when a target device contained in a capturedimage performs an operation corresponding to first action information ofthe first target device, activating a connection between the firsttarget device and the mobile terminal.
 4. The mobile terminal of claim2, wherein the controller is further configured for: transmitting theoperation execution command; and when two target devices contained in acaptured image perform an operation corresponding to first actioninformation of the first target device and an operation corresponding tosecond action information of the second target device respectively,activating a connection between the first target device and the mobileterminal and a connection between the second target device and mobileterminal.
 5. The mobile terminal of claim 2, wherein the controller isfurther configured for: transmitting the operation execution command;and when two target devices contained in a captured image perform anoperation corresponding to first action information of the first targetdevice and an operation corresponding to second action information ofthe second target device respectively, activating a connection betweenthe mobile terminal and one of the first and second target devicesselected by a user.
 6. The mobile terminal of claim 5, wherein theterminal further includes an input interface for receiving an input fromthe user, wherein the controller is further configured for: receiving aninput for selecting one of the first target device and the second targetdevice; and when an operation of the selected target device correspondsto the first action information, activating a connection between themobile terminal and the first target device.
 7. The mobile terminal ofclaim 2, wherein the controller is further configured for: transmittingthe operation execution command; receiving first action information fromthe first target device and receiving second action information from thesecond target device; and when a target device contained in a capturedimage performs an operation corresponding to the first actioninformation, determining that the target device contained in thecaptured image is the first target device.
 8. The mobile terminal ofclaim 7, wherein the first action information includes at least one ofan optical indication presence or absence, an optical indication color,an optical indicating pattern, or an image pattern of an opticalindicator.
 9. The mobile terminal of claim 2, wherein the controller isfurther configured for: transmitting the operation execution command;receiving first action information from the first target device andreceiving second action information from the second target device; whenone of two target devices contained in a captured image performs anoperation corresponding to the first action information, and the otherthereof performs an operation corresponding to the second actioninformation, determining that the one target device is the first targetdevice and the other target device is the second target device.
 10. Themobile terminal of claim 1, wherein the controller is further configuredfor: when first and second target devices among the plurality of targetdevices have the same shape information and when a distance of the firsttarget device is equal to a distance of the second target device,displaying an action request; and when action information correspondingto the action request is received from the first target device,activating a connection between the first target device and the mobileterminal.
 11. The mobile terminal of claim 10, wherein the actionrequest includes at least one of a button pressing operation of thefirst target device or a control operation of at least one component ofthe first target device.